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difference  equations  have  been  formulated  for  the  purpose  of  forecasting  errors 
as  a function  of  time. 
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1.  MAN-MACHINE  INTERFACE  ANALYSIS 


1.1  INTRODUCTION 

Information  gained  in  evaluating  and  solving  man-machine  interface 
problems  that  occur  in  complex  communications  systems  is  extremely  important 
to  systems  engineers  committed  to  the  mission  of  the  design  and  fabrication 
of  future  generations  of  equipment.  Sophisticated  systems  of  command  and 
control  , computer-aided  man-in-the-loop  systems  (e.g.,  manned  spacecraft), 
human  response  to  audio  and  visual  displays,  pattern  recognition,  man- 
computer  languages,  and  many  other  facets,  are  of  concern  where  an  operator 
must  perform  a control  task,  or  decision  task.  At  present  there  is  some 
ongoing  work  oriented  towards  man-machine  interfaces  which  span  the  projected 
needs  of  the  armed  forces,  [1],  [2],  [3],  [4],  [5  ],  [6  ]•  Work  in  progress 
by  many  agencies  generally  deals  with  evaluation  of  complex  system  interfaces, 
assessment  of  operator  performance  capabilities  for  a wide  variety  of  tasks 
(including  performance  as  a function  of  ambient  light  alone),  analysis  of 
manual  functions  into  tasks,  analysis  of  human  control  functions,  and  the 
physical  and  psychological  characteristics  which  affect  the  assessment  of 
operator  performance  capabilities.  Much  of  the  ongoing  work  concerns  the 
psychological  and  physiological  aspects  of  command  and  control , -weapons 
systems,  logistics,  and  communications. 

Examination  of  ongoing  research  in  these  areas  indicate  that  there  is 
no  clear-cut  procedure  to  evaluate  the  human  subsystem  in  sophisticated 
communications  systems  or  the  effects  of  environmental  stress  on  operator 
performance.  Army  communications  requirements  in  a tactical  situation  often 
require  24-hour  operations.  Personnel  are  required  to  work  either  on 
standard,  or  unpatterned,  and  freauently  extended  duty  schedules  in  a variety 
of  environments,  each  characterized  by  multiple  stresses  occurring  in  a 
random  manner.  In  an  effort  to  seek  novel  measures  of  man-machine  interfaces 
which  occur  in  communications  systems  and  to  enhance  the  design  of  future 
families  of  equipment,  this  section  will  address  teletype  operator  perform- 
ance as  the  environmental  factors  of  ambient  light  and  acoustic  noise  are 
varied.  The  discussion  will  center  around  an  experiment  conducted  at 
Fort  Monmouth,  New  Jersey,  during  April  and  May,  1975.  The  experiment  was 
designed  to  answer  the  following  questions: 
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’ ' Is  there  a sianificant  deterioration  of  operator  performance 
(committed  errors)  as  tne  environmental  -‘actors  of  jnhivr.-.  igh:  anc  ^jouot -!-• 
Aoise  are  varied? 

ii)  Does  acoustic  noise  affect  operator  performance  more  than  ambient 
light  in  a deteriorating  environment! 

iii)  Is  operator  performance  significantly  different  for  the  optica! 
display  terminal  as  opposed  to  the  tel etypewri ter  terminal? 

iv)  Given  certain  ambient  lignt  and  acoust'c  noise  levels,  can  one 
oredict  the  number  of  transcription  errors  that  will  be  committed  for  a given 
termi nal ? 

v)  Given  the  operating  conditions,  can  one  specify  the  levels  of  light 
and  sound  so  as  to  minimize  the  number  of  committed  errors? 

It  is  an  aim  of  this  study  to  answer  the  above  questions  with  -espect 
to  continqency  table  analysis,  three-factor  analysis  of  variance,  and 
multiple  non-l-inear  -egression. 

In  tne  next  section.  1.2,  the  desion  of  tne  experiment  is  described 
wnich  includes  the  manner  in  which  the  information  was  obtained,  tne  types 
of  eouipment  employed,  and  the  environment  under  which  the  experiment  was 
conduc*ed.  "he  levels  of  the  variables  considered  are  typical  of  those 
encountered  in  3 tactical  environment. 

’'mm  "wo  way-  ’n  wnich  one  can  proceed  to  answer  the  ouestions  wit." 
respect  to  the  dependence  of  tne  environmental  variables  and  tne  number  of 
committed  errors  (operator  performance).  Contingency  taole  analysis  was 
employed  in  the  one  case,  which  is  covered  in  detail  'in  section  1.3;  a compar- 
ison was  made  of  the  second  case  to  the  classical  three-factor  analysis  of 
variance  technigues  as  described  in  section  1.4  . 

An  extensive  search  was  conducted  to  find  the  best  non-linear  regression 
mode!  that  best  characteri zes  the  relationship  between  the  dependent  and 
independent  variables.  The  best  model  was  judged  by  the  criterion  of  minimum 
residual  variance.  Section  1.5  describes  the  -egression  models  tested.  A 
summary  of  results,  conclusions,  and  recommendations  for  "uture  work  is 
presented  in  section  1.6. 


1,2  SESI3K  QP  ~HE  EXPERIMENT 

"he  signiricance  of  acoust'c  noise  and  amb'ent  light  on  ooe-ator  oer^or"- 
ance  was  invest  gated  using  bcth  an  octica1  display  transmission  device  (see 


figure  l.l),  and  a standard  tel etyoewri ter  (see  figure  1.2).  The  visual 
disolay  terminal  is  designed  to  interface  with  computers  or  store-and-forward 
devices.  Primarily,  it  is  a developmental  equipment  intended  to  visually 
oresent  messages  on  a CRT  display  where  an  operator  can  see  and  correct  his 
message  prior  to  transmission. 

"he  experiment  consisted  of  testing  the  transcri ption  accuracy  of  six 
experienced  communications-center  operators  under  16  different  combinations 
of  environment.  Ambient  light  was  varied  at  four  levels,  ranging  from 
24  foot-candles  to  3 foot-candles;  and  acoustic  noise  was  concurrently  varied 
at  four  sound  pressure  levels  ranging  from  55  dBa  to  95  dBa.  Sound  pressure 
level  (SPL),  measured  in  dBa,  is  in  reference  to  .0002  dynes/cm2  and  is 
considered  the  threshhold  of  hearing.  This  reference  is  roughly  equivalent 
to  a leaf  "falling"  on  a quiet  day.  The  55  dBa  level  was  considered  the 
auiet  condition  where  only  the  inherent  noise  from  the  terminal  equipment, 
sound  room  noise,  and  thermal  noise  were  recorded.  The  95  dBa  level  reore- 
sented  an  extremely  annoying  and  distracting  "pink"  noise.  The  noi  se-powe1- 
Qer  unit  frequency  for  this  type  of  noise  is  inversely  proportioned  to 
freouency  over  a specified  range  and  slopes  down  at  3 dB  per  octave  from 
20  Hz  to  20  KHz.  These  characteristics  are  more  common  to  conference-type 
noise  where  the  higher  and  lower  frequency  components  characterize  motor  and 
eouipment  noises.  Pink  noise  was  also  used  because  it  has  relatively  constant 
energy  oer  octave-bandwidth.  The  chosen  ambient  light  levels  of  24,  12,  6, 
and  3 foot-candles,  respectively , represented  successively  deteriorating  room 
lighting  conditions.  Throughout  the  testing,  the  brightness  of  the  optical 
disolay  was  constant. 

For  each  test  the  operator  was  required  to  type  his  name,  treatment 
combination,  and  date  as  oart  of  the  message  (see  figure  1.3).  The  messaaes 
for  the  exoeriment  consisted  of  forty  random-letter  word  groups  of  five 
characters  each.  They  were  derived  through  a random  number  generator  and  an 
alohanumeric  conversion.  Mo  message  was  a duplicate,  nor  were  they  dupli- 
cated by  any  of  the  operators  on  either  terminal  eouipment.  The  random  letter 
format  was  used  so  that  the  operator  could  not  identify  or  recognize  '■outine 
words  and,  therefore,  would  have  to  concentrate  to  avoid  making  transcription 
errors.  The  aim  of  the  exoeriment  was  to  vary  the  environmental  variables 
and  to  observe  the  transcription  accuracy  of  each  operator  utilizing  the 
visual  display  terminal  as  a function  of  time.  The  response  variable, 
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FIGURE  1.1  OPTICAL  DISPLAY  TERMINAL 


FIGURE  1.2  TELETYPEWRITER  TERMINAL 
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accuracy  (number  of  committed  errors),  was  the  measure  of  transcription 
errors  that  each  operator  coimiitted  per  four-second  interval.  The  errors 
considered  were  the  following: 

1)  transposition 

2)  missing  letter 

3)  extra  letter 

4)  incorrect  space 

5)  extra  line  feed 

6)  missing  word  groups 

7)  wrong  letter 

3)  line  out  of  sequence  (skipped  line  inserted  after  detection) 

9)  word  group  out  of  sequence 

The  results  were  compared  to  an  acceptable  operator  norm,  i.e.,  typing 
a message  format  on  a standard  teletype  terminal  under  the  same  conditions. 
Each  operator  was  tested  alone  in  four  sessions,  each  session  having  been 
programmed  for  eight  random  environmental  combinations,  four  for  eacn  term- 
inal equipment  (see  table  1.1).  The  tests  were  alternated  between  the  optical 
display  unit  and  the  standard  teletypewriter.  This  was  done  to  reduce  the 
effects  of  learning.  A thirty-minute  familiarization  period  was  given  each 
operator  prior  to  the  tests,  and  a standard  ’ instruction  sheet  was  distributed 
during  this  period  to  insure  uniform  orientation  with  the  equipment  and  with 
the  purpose  and  procedure  of  the  exoeriment. 

1.3  CONTINGENCY  TABLE  ANALYSIS  [7  ],  [8  ] 

The  following  analyses  will  attemDt  to  answer  whether  or  not: 

i)  acoustic  noise  affects  committed  errors  more  than  ambient  light 
level , 

ii)  there  is  a difference  in  committed  errors  when  the  optical  display 
terminal  is  used  as  opposed  to  the  teletypewriter, 

iii)  operator  performance  deteriorates  significantly  as  the  light  and 
sound  levels  are  varied. 

Specifically,  the  purpose  of  this  analysis  is  to  show  how  the  variation 
of  light  and  sound  levels  affects  the  number  of  transcription  errors  made  by 
communications  operators  using  two  different  tactical  communications  termi- 
nals. 
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TABLE  1.1 


TREATMENT  SCHEDULE 

PER  OPERATOR 

Session 

Run 

Environmental  Treatment* 
Combinations 

Optical  Display  Teletype 

Terminal  Termina 

I 

1 

1 ,4 

3,1 

2 

4,3 

4,4 

3 

3,2 

2,2 

4 

2,1 

1 ,3 

II 

5 

3,1 

4,1 

6 

4,4 

1 ,2 

7 

/ 

2,2 

3,4 

3 

1 .3 

2,3 

III 

9 

4,1 

2,4 

10 

1 ,2 

3,3 

11 

3,4 

1 ,1 

12 

2,3 

4,2 

IV 

13 

2,d 

1 J 

i « *■* 

14 

3,3 

^ » 3 

15 

1,1 

3,2 

16 

4,2 

2,1 

Treatment 

= (Ambient  Light  Level 

, Acoustic  Noise  Level ) 

Ambient  Light 

Acoustic 

Level 

Noi  se 

Level 

Value 

Value 

1 2d 

ft-candles 

1 

55  dBa 

2 12 

ft-candles 

2 

70  dBa 

3 6 

ft-candles 

3 

30  dBa 

4 3 

ft-candles 

4 

95  dBa 

8 


A trainee  was  chosen  as  a member  of  the  operator  group  to  provide 
insight  into  the  number  of  committed  errors  he  would  commit  versus  those  of 
the  more  experienced  personnel.  There  was  indeed  a significant  difference 
in  his  performance,  as  will  be  shown  later  in  this  section.  As  a result,  the 
data  generated  by  the  trainee  was  deleted  from  the  final  analysis.  Hereto- 
fore, the  trainee  will  be  referred  to  as  the  "fifth"  replication.  The 
analysis,  therefore,  was  broken  down  into  the  following  major  categories: 

i)  section  1.3.1  - contingency  table  analysis,  all  subjects,  for  the 
teletypewriter  terminal. 

ii)  section  1.3.2  - contingency  table  analysis,  all  subjects,  for  the 
oDtical  display  terminal. 

iii)  section  1.3.3  - contingency  table  analysis  with  the  fifth  replication 
deleted  for  the  teletypewriter  terminal. 

iv)  section  1.3.4  - contingency  table  analysis  with  the  fifth  replication 
deleted  for  the  optical  display  terminal. 

1.3.1  Contingency  Table  Analysis,  All  Subjects,  For  The  Teletypewriter 
Terminal 

The  data  was  formed  into  a two-way  contingency  table  by  summing  the 
number  of  errors  for  each  replication  in  each  of  the  sixteen  cells  (environ- 
mental combinations).  For  ease  of  description,  the  ambient  light  level  will 
be  referred  to  as  factor  A,  and  the  acoustic  noise  level  will  be  referred  to 
as  factor  B.  Thus,  for  the  teletypewriter  terminal,  the  following  hypotheses 
were  tested: 


versus 


Hq:  factor  A is  independent  of  factor  B, 


H.j : factor  A is  dependent  on  factor  B, 


with  the  test  statistic: 


x2  - * 


n . n . 

(n..  - ^)2 

z -U n — 


j»i  i-i  Vn-J 


(a-l)(b-l)  • 
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wnere  a = number  of  levels  of  factor  A, 
b = number  of  levels  of  factor  3, 

t h 

n..  = number  of  observations  in  the  ij  cell, 

1 J th 

n . _ = sum  of  observations  in  the  i row, 

1 +*  h 

n . = sum  of  observations  in  the  j n column,  and 
n = total  number  of  observations. 


The  calculated  test  statistic  resulted  in  x2  = 17.41,  and  the  appropri- 
ate critical  values  or  points  at  various  levels  of  significance  with 
(a-l)(b-l)  = (3)  (3)  = 9 degrees  of  freedom  are: 


a = . 

05, 

x'(9)  = 

16.9 

a = . 

025  , 

X’(9)  = 

19.0 

a = . 

01  , 

<Z (9)  = 

21.7 

Therefore,  one  rejects  the 

null 

hypothes' 

is  at  the  a = . 

.05  level . 

To  determine  the  level 

or 

levels  of 

either  factor 

for  which  the  number 

of  errors  is  minimum,  the  row  and  column  totals  of  either  the  entire  table 
or  aDprooriate  partitions  should  be  considered.  Considering  factor  A alone, 
from  the  following  hypotheses: 

Hq:  row  sums  are  eoual  , i.e.,  P, _ = P^.  = P,.  = P^. 

versus 

: row  sums  are  unequal,  i.e.,  P^.*  * P3.  t P4. 

^ h 

where  P._  is  the  probability  of  an  error  in  the  i row;  and  using  the  test 
statistic: 

a (n  it  - n/a)2 

x*  = F7a  ^ x2(a-l)’ 

the  calculated  value  was  found  to  be  x2  = 10.43.  In  this  case,  the  critical 
points  at  various  levels  of  significance,  a,  with  a-1  = 3 degrees  of  freedom 
are: 
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i 


| 


05, 

x(3) 

= 7.31 

025, 

X(3) 

= 9.35 

01, 

X(  3 ) 

= 11.3 

This  indicates  quite  strongly  that  there  is  a difference  in  the  levels  of 
factor  A with  respect  to  the  number  of  errors.  This  further  indicates  that 
there  exists  a level,  or  group  of  levels,  that  results  in  a minimum  number 
of  committed  errors.  Through  a similar  process,  the  row  totals  were  compared, 
one  at  a time,  to  all  the  others  to  determine  at  which  level  the  most  signif- 
icant difference  occurred.  The  test  and  the  test  statistic  were  similar  to 
the  above  with  a=2  being  the  number  tested  in  this  case.  When  comparing  the 
smallest  row  total  to  the  largest,  the  calculated  x2  = 10.08  with  a-1  degree 
of  freedom.  A strong,  significant  difference  was  observed  in  this  case. 

When  considering  the  smallest  row  total  and  the  second  largest,  v2  = 5.22 
with  1 degree  of  freedom,  which  indicated  a significant  difference  at  the 
x = .025  level.  Finally,  when  considering  the  smallest  row  total  and  the 
second  smallest,  x2  = 2.83  with  1 degree  of  freedom,  which  strongly  indicated 
no  significant  difference  in  the  number  of  errors  committed.  Thus,  one  can 
conclude  that  at  3 foot-candles  and  6 foot-candles  of  illumination,  the 
number  of  transcribed  errors  are  minimum. 

Now,  considering  factor  3 alone,  the  following  hypotheses  were  formu- 
lated: 


versus 


Hq.-  column  sums  are  equal,  i.e.,  = P ^ ' p.-,  = p.4 


H, : column  sums  are  not  equal,  i.e.,  ? , i P , i P , « P 


; th 


•3  T -4  * 


where  is  the  probability  of  an  error  in  the  j column.  Using  the  test 
statistic: 


X2  = 2 


<".j  - F>* 


j=l  n 
b 


(b-l)  ’ 


yielded  a calculated  value  of  x2  = 13.99  with  (b-l)  = 3 degrees  of  freedom, 
which  strongly  indicated  a significant  difference  in  the  levels  of  factor  B 
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at  the  3 = .05  level  with  resoect  to  the  number  of  errors.  Again,  this 
indicates  that  there  is  a level,  or  grouD  of  levels,  at  wnich  the  number  of 
committed  errors  is  minimum.  It  was  further  observed  that  only  she  level 
with  the  least  number  of  errors  compared  to  that  with  the  most  errors  snowed 
any  significant  difference,  or  acceptance  of  H^.  Therefore,  tne  conclusion 
for  the  standard  teletypewriter  terminal  is  that  the  factors  of  ambient  light 
and  acoustic  noise  are  dependent,  and  that  the  minimum  number  of  committed 
errors  occur  when  the  ambient  light  is  at  or  above  6 foot-candles,  and,  concur- 
rently, when  the  noise  level  is  at  or  below  30  dBa. 

1.3.3  Contingency  Table  Analysis,  All  Subjects,  For  The  Optical  Display 
Terminal 


For  the  optical  display  terminal,  the  procedure  oescribed  in  section 
1.3.1  was  again  duplicated  with  the  hypotheses  formulated  as: 


versus 


Hg : factor  A is  independent  of  factor  3 


H.|  : factor  A is  dependent  on  factor  3. 


The  calculated  test  statistic  for  this  case  was  - 3.  ‘ with  '?  decrees  pf 

freedom,  which  strongly  indicated  the  independence  of  the  two  factors 
(acceptance  of  the  null  hypotheis  at  the  3 = .05  level).  Continuing  on,  for 
factor  A alone,  i.e.,  the  significance  of  the  levels  of  A on  operator 
performance,  the  appropriate  hypotheses  are: 

H^:  row  sums  are  equal;  P^ _ = D?  = = P^ 

versus 

H, : row  sums  are  uneoual;  P,  £ P„  * P.  * P. 

I I • c • i • 4 • 

The  calculated  test  statistic  was  = 3.17  with  3 degrees  of  freedom,  which 
ind-cated  no  significant  difference  in  the  levels  of  factor  A at  * = .35. 
This  shows  that  varying  the  levels  of  factor  A (ambient  light)  does  not 
sigrficantly  affect  the  number  of  committed  errors. 
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For  factor  B alone,  the  following  hypotheses  were  tested: 

Hq : Column  sums  are  equal,  i.e.,  = P 2 = P.3  = p.4 

versus 

: Comumn  sums  are  unequal,  i.e.,  P ^ f P . 2 t P.3  t P.4 

The  calculated  value  of  the  test  statistic  obtained  was  x2  = 14.29  with 
3 degrees  of  freedom.  This,  of  course,  indicated  a strong  difference  in  the 
levels  of  factor  B at  a = .05  with  respect  to  the  number  of  committed  errors, 
and  further  indicated  a level,  or  group  of  levels,  at  which  a minimum  (or 
maximum)  number  of  errors  occurred. 

To  detect  which  of  the  levels  is  most  significant,  the  following  tests 
were  made: 

Hg:  Least  column  sum  equal  to  the  largest  column  sum,  = P ^ 

versus 

H.j  : Least  column  sum  unequal  to  the  largest  column  sum,  P^  * P 4 

In  this  case,  the  calculated  test  statistic  x2  = 10.24  with  1 degree  of 
freedom  indicated  a significant  difference  between  these  sound  levels,  i.e., 
55  dBa  and  95  dBa  for  a = .05.  Repeating  this  procedure  again  for: 

Hq.-  Least  column  sum  equal  to  second  largest  column  sum,  P.-j  = P , 

versus 

: Least  column  sum  unequal  to  second  largest  column  sum,  P 1 * P 3 

The  test  statistic,  x2  = 1-41  with  1 degree  of  freedom,  indicated  no  signif- 
icant difference  in  the  number  of  committed  errors  at  the  ot  = .05  level. 

This,  of  course,  implies  no  difference  in  the  effects  of  these  levels,  i.e., 
55  dBa,  70  dBa,  and  80  dBa.  The  conclusion  of  this  series  of  tests  was  that 
the  factors  were  independent,  with  the  minimum  number  of  errors  occurring 
when  the  level  of  acoustic  noise  was  at  or  below  80  dBa.  The  level  of  light 
did  not,  however,  affect  the  number  of  errors  produced.  This  was  apparently 
due  to  the  fact  that  the  human  eye  is  highly  adaptable  to  deteriorating 
light  conditions,  at  least  to  the  3 foot-candle  level. 

1,3,3  Contingency  Table  Analysis  With  5th  Replication  Deleted  For  The 
Teletypewriter  Terminal 

What  remained,  then,  was  to  show  the  effect  of  the  trainee  (fifth 
replication)  on  the  results.  Thus,  the  above  series  of  tests  were  again 
duplicated,  with  the  fifth  replication  deleted,  as  follows: 
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t-U:  factor  4 independent  of  factor  B 


versus 

-i-j  : factor  A not  independent  of  factor  3, 

for  which  tne  test  statistic  \z  = 6.42  with  9 degrees  of  freedom  indicated 
acceptance  of  Hr  at  the  a = .05  level;  that  is,  the  two  factors  are  independ- 
ent. Testing  again  with  respect  to  the  levels  of  ambient  light1 

Hq : there  is  no  difference  in  row  totals,  p.  _ = p7.  = P7  = P<1_ 

versus 

H-| : there  is  a difference  oetween  row  totals,  P,  _ * P7.  * P^.  * . 

tne  test  statistic  x~  ' 9-4  with  2 degrees  of  freedom  indicated  a '■'ejection 
of  H,  at  a level  of  significance,  :t  - .05.  Thus,  there  is  a r3nge  of  levels 
at  w.nich  the  number  of  er-ors  is  minimum.  For  the  soecfic  levels  of  A; 


: there  is  no  difference  between  the  smallest  and  largest  row 
totals,  P, . = Pd- 

versus 

H, : there  is  a difference  between  the  smallest  and  largest  row 
totals , P, . = p4. 


The  test  statistic  x2  = 6-49  with  1 degree  of  freedom  indicated  a significant 
difference  at  i = .05. 

Comparing  the  smallest  and  second  largest  row  totals,  P-j , and  P,  , 

X2  = 1.382  with  1 degree  of  freedom,  indicated  no  difference  at  a = .05, 

i.e.,  P,  = P,  . Thus,  the  ranoe  of  levels  at  which  the  committed  errors  are 
1 • 3 • 

minimum  is  6-24  foot-candles. 

To  investigate  the  error  behavior  with  respect  to  acoustic  noise,  we 

have  • 
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there  is  no  difference  between  column  totals,  P , = P - 
P = P ' i 

• 3 -4 


versus 


H,  : there  is  a difference  between  column  totals,  P , * P , f 
P i p • \ • c 

•3  1 -4 

In  this  case,  the  test  statistic,  x2  3 31.29  with  3 degrees  of  freedom, 
indicates  that  there  is  a significant  difference  at  a = .05  between  the 
levels  of  factor  B (column  sums)  with  respect  to  the  number  of  committed 
errors.  This  again  affirms  that  there  exists  a range  of  levels  at  which  the 
number  of  errors  is  minimal.  Testing  for  these  specific  levels,  we  again 
compare  the  smallest  column  total,  P y to  the  largest,  P Here,  x2  3 
23.35  with  1 degree  of  freedom,  indicates  there  is  a significant  difference 
at  a = .35.  Testing  again  for  the  smallest  column  total  and  the  second 
largest,  P,,  X2  3 20.15  with  1 degree  of  freedom.  This  indicates,  as  before, 
that  P _ i * P ^ at  a = .05.  Finally,  testing  for  the  difference  between  the 
smallest  column  total,  P ^ , and  the  second  smallest,  P.,,,  x2  3 3.34  with  1 
degree  of  freedom.  Here,  H^  is  accepted  at  the  a = .05  level.  Thus,  there 
is  no  significant  difference  between  them;  that  is  P ^ = P This  means 
that  the  range  of  level  B at  which  the  number  of  errors  is  minimum  occurs 
between  55  dBa  and  70  dBa. 

1.3.4  Contingency  Table  Analysis  With  5th  Replication  Deleted  For  The 


Ootical  Display  Terminal 


A similar  procedural  aporoach  was  implemented  to  study  the  optical 
display  terminal.  One  begins  by  testing  the  hypothesis; 


versus 


H^:  factor  A is  independent  of  factor  B 


H-| : factor  A is  dependent  on  factor  B 


the  test  statistic  x2  3 14.84,  with  9 degrees  of  freedom,  indicates  that 
factor  A is  independent  of  factor  B (accept  H^)  at  a = .05.  Next, 
considering  the  levels  of  factor  A alone,  we  test: 


15 


'he  calculated  test  statistic  x2  = 2.96  with  3 degrees  of  freedom,  indicates 
no  significant  difference  in  the  levels  of  factor  A at  the  x = .05  level. 

This  means  that  varying  the  level  of  ambient  light  did  not  cause  the  number 
of  committed  errors  to  vary  significantly  as  one  would  expect.  The  test  to 
define  the  range  of  factor  A levels  is,  therefore,  not  necessary  since  the 
orevious  test  indicated  acceptance  of  the  null  hypothesis. 

Mow,  testing  the  hypotheses  with  respect  to  the  levels  of  factor  3 alone, 
we  nave: 

there  is  no  difference  between  column  totals 

versus 

H, : there  is  a difference  between  column  totals 

i 

In  this  case,  x2  = 12.78  with  3 degrees  of  freedom  indicates  a differ- 
ence between  the  levels  with  respect  to  the  number  of  transcribed  errors  at 
the  x = .05  level.  This,  of  course,  indicates  that  tnere  is  a level,  or 
grouo  of  levels,  at  which  the  numoer  of  errors  is  minimal.  To  define  tnese 
levels  (or  range),  the  following  hypotheses  were  tested: 

H0:  there  is  no  difference  between  the  smallest  and  largest  column 
J totals,  P _ i = P-(1 

versus 

H, : there  is  a difference  between  the  smallest  and  largest  column 
1 totals,  P.-j  4 P d 

Here,  x2  = 13.25  with  1 degree  of  freedom  indicates  there  is  a signifi- 
cant difference  at  x = .05.  The  comparison  of  the  smallest  and  second 
largest  column  totals,  P.-j  and  P ^ where  x2  - 2.62  with  1 degree  of  freedom, 
indicates  acceptance  of  HQ  at  the  a = .05  level.  This  further  defines  the 
range  of  levels  at  which  the  minimum  number  of  errors  are  committed,  namely, 

55  dBa  to  30  dBa. 
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Thus,  from  this  series  of  tests  we  conclude  that  the  factors  of  ambient 
light  and  acoustic  noise  are  independent,  with  the  minimum  number  of  errors 
occurring  when  the  sound  level  is  below  80  dBa.  The  level  of  light,  however, 
does  not  significantly  affect  operator  performance.  The  human  eye  is  highly 
adaptable  to  deteriorating  light  conditions,  at  least  to  the  3 foot-candle 
level . 

The  conclusion  evident  from  sections  1.3.3  and  1.3.4,  in  comparing  the 
results  with  those  of  sections  1.3.1  and  1.3.2  is  that,  due  to  the  discrep- 
ancies in  the  results,  the  trainee  (operator  with  only  three  months 
experience)  should  not  be  considered  in  the  analysis  of  the  performance  of* 
experienced  operators.  Therefore,  the  conclusions  of  sections  1 .3.3  and 
1.3.4  should  be  considered  the  most  valid. 


1.4  ANALYSIS  OF  VARIANCE 

In  view  of  the  fact  that  the  data  was  limited  in  the  number  of  observa- 
tions, the  contingency  table  analysis  was  implemented  to  answer  the  relevant 
questions  without  having  to  place  a probability  structure  on  the  observed 
information.  Secondly,  although  the  assumptions  of  normality  cannot  be 
justified  with  the  limited  amount  of  information  available,  an  analysis  of 
variance,  [9],  [10],  was  performed  on  the  man/machine  interface  data  so  as 
to  obtain  comparable  results  on  the  Questions  posed  in  section  1.1. 


1.4.1  Two-Factor  Analysis  of  Variance 


Considering  the  tel etypewri ter  terminal  first,  the  initial  model 


Y = |J  + Ct .+  6 • + Y , 

1 J ijk 


was  taken,  where: 


Y = the  dependent  variable  which  represents  the  number 
of  transcribed  errors, 

u = the  overall  mean  effect, 

a . = the  effect  of  the  levels  of  factor  A (ambient  light) 
for  the  i th  level , 

3 . = the  effect  of  the  levels  of  factor  B (acoustic  noise) 
J for  the  level, 
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v.  = the  effect  of  the  interaction  of  A and  B, 

= the  experimental  error;  that  ;s,  the  extent  tc  wnich 
the  observea  data  anc  the  general  .node!  disagree  for 
k reDl ications . 

,*rm  this  formulation,  and  from  the  tel et/oewr' ter  terminal  data,  the 
o' lowing  ANOVA  table  was  obtained: 


df. 

Sum  of  Square,  S.S.  . 

M.S.  - Mean  Sauare  = ^ ' s£' 

factor  A 

| 

a-1  = 3 

55.9375 

18.6453 

factor  3 

D-l  = 3 

75.0375 

25.0125 

Interact! on 
AB 

(a-l)(b-l)  = 9 

107.5125 

11.9458 

of  **0  Y* 

abf  k-1 ) = 64 

1312 

20.3 

total 

abk-1  = 79 

1 550.4375 

ci rst , 

testing  for  the 

signi “'cance  of  the  i 

nteraction  AB,  the  following 

hypotheses  were  formulated: 

Hn : v = 0 against  H- : v.,  = 0. 

^ ■ J 1 • U 

iD crocr-’ate  test,  statistic  / i e 1 d s : 


* .5327  and  F ( a)  » F f .05)  = 2.04. 

'j:  fa-1  )(b-l ) , ab(k-l ) 9,64 

Thus,  the  null  hypothesis  that  the  interaction  of  A and  3 is  not  signif- 
icant at  -x=  .05  is  accepted.  This  shows  that  the  model  is  additive  as 
ODCOsed  to  the  possibility  of  being  multiplicative.  One  can,  therefore, 

corrDine  the  y..,  and  c...  . This  results  in  a model  of  the  form: 
i J i J k 


Mow,  testing  for  the  significance  of  the  levels  of  factor  B,  namely  3-, 

J 


Hq:  Bj  = 0 versus  : 3j  f 0, 

MCD 

we  have  jrjjf-,  = 1.286  and  F (a)  3 2.74.  Thus,  we  accept  the  null  hypothsis 

3,73 

at  the  a = .05  level  cf  significance.  This  result  shows  that  varying  the 
levels  of  factor  B will  not  significantly  affect  the  number  of  comnitted 
errors.  Now,  since  S.  is  not  significant,  one  may  combine  the  B,-  and  e’--,,. 
This  results  in  a new  model: 


Y = u + a.  + e".  .. 


where 


,il  _ _ ! 


ijk 


...  + 3-  = S • + y • • +£.., 
ijk  J J ijk 


and  the  SSE"  = SSB  + SSAB  + SSE  = 1494.55  with  (a-1 ) (b-1 ) + b-1  + ab(k-l) 
= 76  degrees  of  freedom. 

Testing  for  the  significance  of  the  levels  of  factor  A;  that  is 


Hq:  a..  = 0 versus  H, : a^  £ 0, 


we  have 


MSA 
M5T  ' 


.948  and  F (a)  7 2.74,  which  implies  that  Hn  must  be 

3,76  J 


accepted  at  the  a = .05  level  of  significance.  This  shows  that  varying  the 
levels  of  factor  A will  not  significantly  affect  the  number  of  errors 
produced.  The  apparent  conclusion  of  these  tests  is  that  the  model  which 
characterizes  the  differences  in  the  number  of  errors  in  each  cell  is: 


where 


Y 


y 


+ 


£ 


ijk 


e - =ai 


ijk 


+ 3j 


ij 


'ijk' 


Repeating  the  above  procedure  for  the  optical  display  terminal  with  the 
same  initial  model,  hypotheses,  and  parameters  as  previously  defined,  the 
initial  ANOVA  table  is: 
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Decrees  of 

“»~o©rjr*rn  1 


Sums  of 


'''ean  Sauares, 


-actor  A 

3 

13.5375 

c . t>c 

ractor  3 

“) 

88.4375 

29.479 

I n w"  cr. 

3 

Z 7 

? . 43/ 

3ri-or 

1 098 . 

1 " . 1 56 

To  ta  1 

"9 

1264.1875 

Testing  for  the  significance  of  the  interaction,  we  nave: 

H : y = Q versus  H, : ■ - 2 

0 ‘C  1 

Mean 

ne  calculated  test  statistic  in  this  case  is:  t.-  ■ = 0.375  and 

■lac 

a)  = 3.04.  Thus,  we  accept  H„  at  the  x = .05  level  of  significance.  This 
3, 5<i  J 

again  indicates  that’ the  model  is  additive,  ana  the  interaction  term,  v.  . , 

id 

can  be  combined  with  Now,  the  revised  model  is  written  as: 


"*  i " j ' i i l< 

.•mere  : „;,k  = 7.  . - anc  the  resulting  2SE ' = '415.5125  witn  73  degrees 

•J  J yj  ' 

C*  'roo/jom. 

Next,  we  consider  tne  significance  cf  acoustic  noise;  that  is, 

H^:  3^  = 0 versus  H.,  : 3.  * 0 

, vicr 

ne  ca.cuiated  test  statistic  *s  = 1.516  and  F (a)  = 2.74.  Ther“fore. 

•',t  3,73 

we  acceDt  H at  a = .05.  ~he  ’nd'cation  is  that  varying  the  levels  of 
factor  5 will  not  significantly  affect  the  number  of  errors.  Thus,  combining 
£.  and  a1...,.  yields  the  revised  model: 


'i  ~ ijk 


where  s'  1 ...  = 5,  * y.  . * s,  . and  the  5SE ' ' - 1244.55  with  76  degrees  of 

' J J J . X J 

► v-aorlQ^  _ 


Finally,  we  test  the  levels  of  factor  A,  the  a.;  that  is, 

H • ex-  = 0 versus  H-j : a.  f 0 

fyir  A 

one  obtains  the  test  statistic:  r^-p,  , = 0.3997  and  F (2)  3 2.74.  Thus,  we 

M5E  3,76 

accept  Hg  at  the  a = .05  level  of  significance.  Here  again,  as  with  the  tele 
typewriter,  it  is  indicated  that  varying  the  levels  of  factor  A will  not 
significantly  affect  the  number  of  errors.  The  conclusion  reached  from  these 
tests  is  that  the  model  which  describes  the  difference  in  the  sums  of  errors 
in  each  cell  for  both  the  optical  display  terminal  and  the  teletypewriter 
terminal  is: 


This  means  that  there  is  no  apparent  effect  on  operator  performance  for  either 
terminal  as  the  levels  of  ambient  light  and  acoustic  noise  are  varied  within 
the  chosen  ranges.  It  should  be  noted  that  the  criteria  for  analysis  of 
variance  (that  the  observations  in  each  cell  are  normally  distributed  and 
that  the  variances  in  each  cell  are  homogeneous),  were  not  substantiated. 
Therefore,  the  results  of  sections  1.3.3  and  1.3.4  shou-ld  prevail. 

In  view  of  these  results,  the  next  step  in  the  everall  analysis  was  to 
identify  and  to  sort  out  any  additional  factors  that  may  have  affected  the 
experimental  error.  The  following  sub-section  addresses  the  incorporation  of 
the  most  logical  third  effect,  namely,  performance  differences  between 
operators. 

1.4.2  Three-Factor  Analysis  of  Variance 

Since  there  was  an  appreciable  difference  in  the  number  of  committed 
errors  among  the  subjects  tested,  an  additional  variable  was  defined,  namely, 
operator  difference.  Thus,  for  this  analysis  a three-factor  analysis  of  vari- 
ance, [11]?  was  performed  with  factors  A,  B,  and  C,  representing  light,  sound, 
and  operator  difference,  respectively.  The  respective  levels  of  each  main 
factor  a = 4 (the  levels  of  ambient  light),  b = 4 (the  levels  of  acoustic 
noise),  and  c * 5 (the  number  of  operators,  excluding  the  trainee). 
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In  a completely  randomized  experimental  desman,  che  ■"odel 
three-factor  analysis  is  given  oy  [II]: 


'"or  the 


'ijkl 


(1.1, 

i = 1,  2,  ...a;  j = 1,  2,  ...b;  k = 1,  2,  ...c;  and  1 = 1,  2,  ...n,  where 

is  the  dependent  variable  representing  the  number  of  errors,  u is  the 
overall  mean  effect,  a-,  3-,  and  y,  are  the  main  effects;  (a6) • • , (ay),,,  ana 

I J K 1 J • K 

(By).,  are  the  two-factor  interaction  effects  that  have  the  same  interpreta- 
tion  as  in  a two-factor  experiment.  The  term  (a  Sy)^  is  called  the  three- 
factor  interaction  effect,  namely,  a term  that  represents  a nonadditivity  of 

the  (a6)..  over  the  different  levels  of  the  factor  C.  The  sum  of  all  main 

’ j 

effects  is  zero  and  the  sum  over  any  subscript  of  the  two-  and  three-factor 
interaction  effects  is  zero.  In  many  situations,  these  higher-order  inter- 
actions are  insignificant  and  their  mean  souares  reflect  only  random 
variation. 


The  general  philosoohyof  the  analysis  is  the  same  as  that  for  a one-  or 
two-factor  experiment.  The  sum  of  squares  is  partitioned  into  eight  terms, 
eacn  representing  a source  of  variation  from  which  one  obtains  independent 
estimates  of  the  common  variance,  o2,  when  all  the  main  effects  and  inter- 
action effects  are  zero.  If  the  effects  of  any  given  factor  or  interaction 
are  net  all  zero,  then  the  mean  souare  will  estimate  the  error  variance  plus 
a component  due  to  the  effect  in  auestion. 

The  computational  procedure  for  obtaining  the  sums  of  souares  in  a 
three-factor  analysis  of  variance  requires  the  following  notation: 


sum 

of 
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aben  observa 

ti  ons 

• i . . . 
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f h 
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. i • • 

= 
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of 

the 
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1 J, 
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sum 
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1 evel 
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A and  the 
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Ti-k- 

= 

sum 

of 
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observations 
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the 

1th 

level 

of 

A and  the 

k”^1  level  of  C 

1 • jk  • 

= 
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of 
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.th 

J 

1 evel 

of 

- th 

3 and  k 

level  or  C 
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= 

sum 

of 
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observations 

for 
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( i jk 

) l,n  treatment  combination. 
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c 

Total 
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The  sum  of  squares  are  computed  by  using  the  following  computational 
formulas,  [ll]: 

a b c n 

SST  = I Z I E y; 


T? . . . 


i=l  j=l  k=l  1=1 


ijkl  a ben 
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■ E T2J 
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and  the  SSE,  as  usual,  is  obtained  by  subtraction.  The  computations  in  an 
analysis-of-variance  problem  for  a three-factor  experiment  with  n replica- 
tions are  summarized  in  the  following  table,  [11]: 
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THREE-WAY  ANOVA 


Source  of 
Variation 

Sum  of 
Sauares 

Degrees  of 
Freedom 

Mean 

Square 

Computed 

Main  Effects 

A 

SSA 

a-1 

1 

s2 

s2 

3 

SSB 

b-1 

1 

! 

! 

1 S‘ 

z 

1 

" 2 

s2 

s2 

ssc 

c-1 

i 

! s2 

3 

n 

S'* 

= — L 

S *" 

Two-factor 

Interactions 

AB 

SS(AB) 

(a-1 ) (b-1 ) 

l 

1 

1 s2 

j? 

s2 

s2 

AC 

SS(AC) 

(a-1 ) (c-1 ) 

| , 

3 

s2 

s2 

3C 

SS(SC) 

(b-1 ) (c-1 ) 

s2 

• 6 

j 

5 

s2 

_ 5 

s2 

Three-factor 

Interactions 

ABC 

SS(ABC) 

(a-1 ) (b-1 ) (c-1 ) 

i 

! 

s2 

i 

7 

s2 

s2 

Error 

SSE 

abc(n-l ) 

- s2 

i 

Total 

SST 

abcn-1  i 

For  a three-factor  analysis  with  a single  replicate,  one  sets  n=l  and 
uses  the  ABC  interaction  for  the  error  sum  of  squares.  In  this  case,  one 
assumes  that  the  ABC  interaction  is  zero  and  that  SS(ABC)  represents  varia- 
tion due  only  to  experimental  error  and  is,  therefore,  the  estimate  of  the 
er~or  variance. 


1 
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Using  this  methodology,  the  following  ANOVA  table  was  obtained  for  the 
teletypewriter  terminal  data: 


Source  of 
Variation 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 

Square 

Computed 

A 

55.938 

3 

18.646 

1.912 

B 

75.037 

3 

25.012 

2.564 

C 

813.674 

4 

203.418 

20.854 

AB 

107.512 

9 

11.946 

1.225 

AC 

125.020 

12 

10.418 

1.068 

BC 

22.126 

12 

1 .844 

0.189 

ABC  (error) 

351.161 

36 

9.754 

The  associated  critical  values  of  the  F distribution  for  the  y = .05 
level  of  significance  are: 


F (a)  = 2.88 

3.36 

F (a)  = 2.65 

4.36 

F (a ) = 2.16 

9.36 

F (a)  = 2.04 

12.36 


In  this  case,  n=l  replication  per  cell,  the  trainee  was  deleted,  and 
the  error  term,  as  previously  mentioned,  is  the  mean  square  of  the  ABC  inter- 
action term.  One  can  conclude  from  the  computed  f statistics  that  the 
operator  differences  are  indeed  significant,  and  that  light,  sound,  and  the 
second  order  interactions  are  insignificant. 

This  shows  that  the  model  is  additive  with  respect  to  factors  A,  B,  C, 
and  the  interactions,  as  opposed  to  the  possibility  of  being  multiplicative. 
Since  we  are  primarily  interested  in  the  main  factors  A,  B,  and  C,  one  can 
combine  the  (aS)^,  (ay)-b,  (Sy)jk,  and  (aSy)^.  This  results  in  a model 
of  the  form: 
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y = u 


+ 3j  + Yk 


ijk 


wnere 


(a6)ij  + (otY)ik  + (Sv)jk  + (aSY)ijk 


The  new  ANOVA  table  and  new  sum  squared  error  term  are  shown  below: 


Source  of  Variation 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 

Square 

Computed 

A 

75.037 

3 

25.012 

2.349 

3 

55.933 

3 

18.546 

2.124 

C 

313.674 

4 

203.413 

23.168 

Error 

(AB  * 3C  + AC  - ABC) 

505.31  9 

59 

3. 730 

— 

The  associated  critical  values  of  tne  F distribution  for  the  2 = .05 
level  of  significance  are: 

F (a)  = 2.76  and  F (2)  = 2.53 
3,69  4,69 


Thus,  one  can  conclude  that  varying  the  levels  of  ambient  light 
(factor  A),  does  not  significantly  affect  the  number  of  errors  committed, 
while  varying  the  levels  of  acoustic  noise  does  have  a significant  effect  on 
operator  performance.  Therefore,  sorting  out  the  operator  differences 
(which  were  significant  as  expected)  achieved  a decision  comparable  to  the 
contingency  table  analysis  of  section  1 .3.  Now,  for  the  second  case, 
considering  the  optical  display  unit  data,  the  following  ANOVA  table  applies 
(with  the  F distribution  critical  points  the  same  as  for  the  teletypewriter 
da ta ) : 
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Source  of 
Variation 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 

Squares 

Computed 

A 

19.637 

3 

6.546 

0.531 

B 

88.433 

3 

29.479 

2.392 

C 

487.998 

4 

121 .999 

9.899 

AB 

58.110 

9 

6.457 

0.524 

AC 

49.799 

12 

4.150 

0.337 

3C 

116.497 

12 

9.708 

0.788 

ABC(error) 

443.679 

36 

12.324 

The  conclusions  drawn  from  this  information  are  similar  to  those  of  the  first 
case.  With  the  interaction  effects  not  significant  and  thereby  additive, 
the  following  model  was  formulated:  y = u + a.  + 3-  + y,  + s'...,  for  n=l 

' J K 1 J K 

observations  per  cell,  where  s‘iik  = (ctB)^  + (ay)ik  + (3y)jk  + (aSy)iik 

as  in  the  first  case.  The  following  ANOVA  table  was  constructed  for  the  new 
model : 


Source  of 
Variation 

Sum  of 
Squares 

Degrees  of 
-’'"eedom 

Mean 

Squares 

Computed 

A 

19.637 

3 

6.546 

0.676 

3 

38.438 

3 

29.479 

3.045 

r 

487.998 

4 

121 .999 

12.500 

Error 

(AB+AC+BC+ABC) 

668.030 

69 

9.682 

For  the  3 = .05  level  of  significance,  both  operator  differences  and  acoustic 
noise  had  a significant  effect  on  operator  performance.  Therefore,  sorting 
out  the  operator  differences  as  a third  variable  (which  was  again  significant 
as  expected)  achieved  decisions  comparable  to  the  contingency  table  analysis, 
namely,  that  acoustic  noise  significantly  affects  the  number  of  errors 
committed,  while  ambient  light  (within  the  3 to  24  foot-candle  range)  does 
not.  In  view  of  the  results  obtained  through  this  section,  the  next  step  in 
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the  analysis  is  to  develoo  a scheme  to  predict  or  to  precisely  characterize 
operator  performance  as  a function  of  light  and  noise.  To  this  end,  the 
following  section  will  address  a non-linear  regression  technique  specifically 
geared  to  this  type  of  problem. 

1.5  A ,ncn-l:near  regression  ^ooel  ^qr  man-machine  interface 

In  this  section,  an  acceptable  model  [12],  [13],  to  predict  operator 
performance  is  presented  with  the  appropriate  technique  to  determine  the 
environmental  combination  of  ambient  light  and  acoustic  noise  that  generally 
causes  a minimum  number  of  committed  errors.  Various  linear,  multiple 
'inear,  and  non-linear  models  were  tested  for  both  the  optical  display  term- 
inal and  the  teletypewriter  terminal.  What  follows  is  considered  to  be  the 
best  model  for  both  experiments.  The  criterion  used  for  choosing  the  best 
model  was  the  minimum  SSE  (sum  of  squares  error)  where 

SSE  * Z (Y  - Y,)2, 
i = l 

and  Y ■ = observed  errors 
Y.j  = predicted  errors 


1,5.1  Non-Linear  Modeling: 

The  general  model  that  best  describes  the  observed  data  is  of  the  form: 


^ f ;orp 


Y = 


3.  + 3-,  X,  * 3„X„  3.X,  X,  + S.X,2  * S-X2 

U It  c.  c.  d \ L 41  dc. 


+ e6xrx|  ♦ S7xfx2  + 3sxfx|  4 S9xj  4 310x^  4 £. 


(1  .2) 


Y = average  number  of  errors  (operator  performance)  oer 
cel  1 

X,  = ambient  light  level, 

( 

X.  = acoustic  noise  level. 

3.  = model  coefficients,  i =0,  1,  ...  10, 
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£ . = experimental  error,  j = 1,  ....  n (the  extent  to  which 
J 

the  observed  data  and  the  model  disagree,  where  c^'s 
are  independent  and  £ ^ N(0,  a2I)),  and 
n = 16. 

The  estimated  values  of  the  coefficients,  error  variance,  correlation, 
and  appropriate  F statistic  for  both  terminals  are  summarized  in  table  1.3. 

TABLE  1.3  ESTIMATED  PARAMETERS  FOR  NON-LINEAR  MODELS 


Parameter 

Optical  Display 
Terminal 

Teletypewriter 

Terminal 

34.7500 

-7.793 

S, 

1 1 

.5092 

-6.365 

1 

3? 

-1.0840 

1 .018 

! s3 

- .0399 

.1588 

34 

.0359 

.1668 

3«- 

.0137 

- .02055 

36 

.0002373 

- .0007769 

3? 

.001990 

- .004906 

S3 

- .000011 

.00002257 

• 

| 39 

.003293 

.001425 

i 310 
| 

.000053 

.0001133 

SSE 

5.136 

3.389 

s2 

£ 

1.027 

6779 

F(M0DEL) 

2.735 

6.536 

R2~ 

yy 

.8455 

.9289 

R ~ 

yy 

.9195 

.9638 

i 

In  the  case  of  the  optical  display  terminal,  the  F statistic  indicates 


a possible  overabundance  of  variat 


the  cast 


the  teletypewriter 


terminal,  the  small  SSE,  larqe  R:  . and  relatively  small  F statistic,  inai- 

- yy 

cate  an  acceptable  model. 

Mow.  we  begin  to  'nvestigate  tie  possibility  of  reducing  the  aoove  mode' 
consistent  with  the  principle  of  parsimony.  That  is,  we  eliminate  the  vari- 
ables in  the  model  that  do  not  significantly  contribute  to  the  dependent 
variable.  The  procedure  used  to  form  the  reauced  models  was  the  ‘forward 
selection  procedure,"  [ 9 ]?  which  begins  with  the  variable,  X. , that  has  the 

highest  correlation  ox.y.  Next,  the  partial  correlation  coefficients  of  the 

v/ 

remaining  x,  and  y,  p(x.y  1 x . ) , j * i,  are  calculated.  The  x.  with  the 
■j  J * J 

greatest  c( x .y  x.)  is  selected  tc  enter  the  regression  eauation.  This 

process  is  continued,  and  as  each  variable  is  entered  into  the  equation,  the 

multiole  correlation  coefficient  R2  ' and  the  partial  F test  value  for  the 

yy 

most  recent  entry  are  examined.  In  the  first  case,  one  checks  to  assure  a 
relatively  ’nsicnificant  chanae  in  R2%  and,  secondly,  whether  or  not  the 

yy 

inserted  variable  has  taxer  up  a significant  amount  of  variation  over  the 
previous  variaoles  in  the  regression  model,  when  the  partial  F test  becomes 

insianificant  < the  SSE  is  sufficiently  reduced';  and  R2'  is  not  very  differen: 

yy 

from  the  ‘full  model,"  the  process  is  terminated.  The  reduced  model,  there- 
fore, is  that  which  contains  all  significant  variables  ol  is  the  £irst  two 
■ nsi gni £i cant  variables  to  accommodate  any  error  due  to  the  estimates. 

Baseo  on  the  general  model  previously  stated  (see  equation  1.2),  the 
aporooriate  reduced  models  that  characterize  operator  performance  for  both 
terminals  are  as  follows: 

i)  for  the  optical  display  terminal: 


y = 3 


where : 


■’  X X 2 - £’  X 


— 1 H 'J 

- 1 C . o J , 


= -0.1239. 


' . 0C0023 , 


- -G. 0002202. 


Ihaoter  5 
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3 


= 0.000008267, 


with  SSE  = 8.678, 


S2  = 0.7389, 

F(M0DEL)  = 7'783, 

R2"'  = 0.7389, 

yy 

R A = 0.8596 

yy 

ii)  for  the  teletypewriter  terminal: 

Y = 3'  3'  X + S'  X X +3'  X2X  +3'  X3  + 3'  X3  + e 

0 1 1 212  312  4 1 52 


where:  S' 

3 

3' 

2 

3' 

3 

3' 


3' 

with  SSE 

S; 


(MODEL) 

0 2 ^ 


yy 


3.211 , 

-1 . 365 , 
0.03532, 
-0.001288, 
0.002123, 
-0.000004273, 
7.53, 

0.763 

10.5 

0.34 

0.9165 


The  reduced  models  now  provide  the  capability  to  predict  the  number  of 
transcribed  errors,  given  the  desired  combination  of  ambient  light  and 
acoustic  noise.  Further,  they  also  permit  a reasonable  extrapolation  outside 
the  tested  environmental  limits  to  simulate  additional  data. 


1 P n n *•  m a 

v o C . .1  .Cl 


i me  Sour, a -evys 

One  can  now  attempt  to  find  the  light-sound  combination  that  causes  the 
least  numoer  of  errors  to  De  committed.  One  method  of  accomol i sning  this 
task  is  the  standard  method  of  differential  calculus.  This  entails  taking 
the  aerivati  es  of  the  prediction  equation  with  respect  to  X-|  and  X^, 
resDect’ vely , setting  them  eaual  to  zero,  and  solving  the  resultant  system 
of  eauations  simultaneously  for  X,  and  X?  with  the  proper  constraints.  The 
method  used  instead  is  to  simply  evaluate  the  predicted  value  of  Y for 
ordered  oams,  (X-j , X,,},  where  X,  assumes  all  integer  values  from  1 to  26, 
and  X„  assumes  even  integer  values  from  50  to  100.  These  ranges  of  X^  and 
X-  were  chosen  based  upon  the  levels  of  X,  and  X,  used  in  the  experiment. 

“he  orecicted  Y values,  i.e.,  tne  predicted  number  of  errors,  were 
calculated  for  the  envi ronemtnal  combinations  described  in  section  1.2  for 
the  optical  display  data  (using  the  ^educed  model)  to  ootain  the  matrix  of 
table  1.4.  Visual  examination  of  this  matrix  shows  that  the  minimum  numoer 
of  errors,  ’.e.,  4.4,  will  occur  at  a light  level  of  24  £oct-candles  and  a 
concurrent  noise  level  of  54  dBa , or,  if  we  are  willing  to  extrapolate 
sligntly  outside  tne  region  from  wnich  data  has  oeen  obtained,  the  absolute 
minimum,  2.3,  occurs  at  26  foot-canales  and  50  dBa.  Thus,  one  can  conclude 
that  the  minimum  number  of  errors  committed  on  the  ootical  terminal  (in  the 
region  for  wnich  data  was  taken)  occurs  at  the  minimum  sound  and  maximum 
light  combination,  that  is,  24  foot-candies/55  dBa. 

A similar  matrix  of  oredicted  errors  was  computed  for  the  reduced  tele- 
typewriter model,  and  is  shown  in  table  1.5.  In  this  case,  visual  examination 
shows  that  the  minimum  number  of  predicted  errors  occurs  at  a light  level  of 
about  16-17  foot-candles  and  at  a concurrent  sound  level  of  about  55  dBa.  In 
both  cases  (optical  display  and  teletypewriter)  the  results  of  the  minima 
were  exoected.  It  is  to  be  noted,  however,  that  in  a tactical  situation,  the 
environmental  factors  of  ambient  light  and  acoustic  noise  are  far  ^rom 
optimal,  “he  true  worth  of  the  matrices  (predictions)  is  to  show  for  a wide 
variety  of  the  environmental  ^actors.  X,  and  X^,  what  one  can  expect,  that  is, 
how  well  experienced  communicators  will  perform. 
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1.6  SUMMARY  OF  RESULTS,  CONCLUSIONS,  AND  RECOMMENDATIONS 


r 


The  table  on  the  following  page  has  been  formulated  to  surrmarize  the 
analysis  of  the  preceding  sections.  The  indications  from  the  tests  of 
independence  are  clear,  that  the  effect  of  the  trainee  did  in  fact  bias  the 
initial  results.  The  decision,  therefore,  to  delete  the  trainee's  results 
from  the  analysis  was  indeed  valid.  One  can  note  from  the  results,  that  the 
non-parametric  decision-analysis  was  consistent  with  the  parametric  approach 
and  is  considered  extremely  necessary  to  verify  analysis  when  the  data  base 
is  not  large  enough  to  support  the  basic  assumptions  of  normality. 

From  the  data  base,  using  the  two-factor  analysis  of  variance,  no  statis- 
tical significance  can  be  attributed  to  the  effects  of  ambient  light  and 
acoustic  noise  on  operator  performance.  However,  considering  a third  varia- 
ble, the  difference  between  operators  (performance-wise),  and  incorporating 
a three-way  classification  analysis-of-variance,  one  can  conclude  that  oper- 
ator performance  deteriorates  significantly  as  sound  is  increased  to  annoying 
levels  above  80  dBa,  while  the  effect  of  ambient  light  had  little  effect  on 
the  number  of  errors  committed.  This  indicates  the  adaptability  of  the 
operators  to  photic  noise. 

It  is  clear  also,  that  the  operators'  performance  was  somewhat  more 
critical  with  the  optical  display  unit.  The  results  show  clearly  that  oper- 
ator performance  deteriorates  considerably  as  ambient  light  decreases  to 
levels  at  or  below  3 foot-candles  of  illumination  and  as  acoustic  noise 
increases  to  levels  at  or  above  80  dBa.  Contrary  to  what  one  might  expect 
for  the  teletypewriter  terminal,  however,  the  level  of  ambient  light  is  not 
significant  at  all  and  noise  levels  above  30  dBa  are  critical. 

Optimal  and  acceptable  environmental  conditions  for  both  terminals  can 
be  adjudged  from  tables  1 .4.  and  1.5.  They  show  the  number  of  errors  one  can 
expect  for  a full  range  of  light  and  sound  conditions  for  each  terminal.  The 
tables  are  the  result  of  direct  computations  utilizing  the  reduced  non- 
linear models  (see  section  1.5.1  and  1 .5.2)  which  best  describe  the  data. 

The  experiment,  inferences,  and  conclusions  outlined  in  the  preceding 
sections  could  be  strengthened  to  give  more  accurate  results  by  incorporating 
the  following  improvements:  first,  there  were  the  environmental  factors 
which  were  limited  to  ambient  light  and  acoustic  noise,  which  may  not  (in  the 
ranges  specified)  be  the  most  important  factors  influencing  operator  perform- 
ance. Certainly,  operator  performance  is  a function  of  other  stress 
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TABLE  1 .6 


SUMMARY  OF  ANALYSIS 


TYPE  OF  ANALYSIS  AND 

DECISION  AT  THE 

u=  .05  LE 

/EL 

NULL  HYPOTHESIS 

All  Subjects 

Trainee 

Deleted 

Teletype 

Optical 

Teletype 

Ootica 

CONTINGENCY  TABLE  ANALYSIS 

Hq:  Factor  A Independent  of  B 

Reject 

Accept 

Accept 

Accept 

Factor  A Alone  (Ambient  Light) 

Hq : Row  Sums  are  Equal 

Reject 

Accept 

Accept 

Reject 

V Pl-  - P4- 

Reject 

— 

-- 

Reject 

V P1  ■ = P3- 

Reject 

-- 

-- 

Accept 

H • P = p 

O’  1-  Kz- 

Accept 

- - 

— 

— 

Factor  B Alone  (Acoustic  Noise) 

Hq:  Column  Sums  are  Equal 

Reject 

Reject 

Reject 

Reject 

4 • D - p 

O'  ' -1  -4 

Reject 

Reject 

Reject 

Reject 

J ■ 3 = 3 

'O'  -I  -3 

Accept 

Accept 

Accent 

-eject 

H ■ 0 3 

H0 ' ' • 1 = ' - 2 

-- 

-- 

-- 

Accent 

TWO- FACTOR  ANALYSIS  OF  VARIANCE 

Interaction , Hn:  y.  . = 0 

J i J 

Accent 

Accept 

Acoustic  Noise,  H-:  3.  = 0 

J 

Accent 

Accent 

Ambient  Light,  Hq : a.  = 0 

Accept 

Accept 

THREE-FACTOR  ANALYSIS  OF  VARIANCE 

Interactions : 

Hq;  ( ci8 ) -j  j = 0 

Accept 

Accent 

(»Y)ik  = 0 

Accent 

Accept 

V (3v)jk=0 

Accept 

Accept 

Main  Effects: 

Acoustic  Noise,  Hq:  a..  = 0 

Reject 

Reject 

Ambient  Light,  Hq-.  3-  = 0 

Accept 

Accept 

Operators,  Hq:  Yk  = 0 

Reject 

Reject 
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variables,  such  as  training,  temperature,  humidity,  room  configuration, 
fatigue,  message  backlog,  and  even  the  number  of  people  present  in  the  room, 
to  name  a few.  Additional  stress  should,  therefore,  be  created  through 
longer  messages  (fatigue,  training),  and  by  imposing  a message  backlog. 

Secondly,  by  increasing  the  number  of  environmental  levels,  and  also 
the  range  of  levels,  the  sample  size  (number  of  cells)  would  be  increased 
and  thereby  provide  additional  information  on  the  operators'  performance. 
Intuitively,  if  ambient  light  became  extremely  low,  i.e.,  factor  A < 1 foot- 
candle  of  illumination,  extremely  noisy  conditions  should  distract  the 
terminal  operator  to  a much  higher  degree  than  indicated  here,  thus  causing 
a greater  number  of  errors  to  be  coirmitted.  Since  human  operators  all  behave 
in  a random  manner,  and  no  two  are  alike,  the  increased  ranges  and  levels 
should  indicate  the  tolerant  stress  level  one  can  expect  under  deteriorating 
environmental  conditions.  In  this  case  especially,  the  models  obtained  would 
be  strengthened,  producing  a more  accurate  reduced  model.  Consequently,  the 
inference  derived  from  the  models  would  also  be  strengthened. 

Realizing  these  improvements  would,  of  course,  increase  the  expense  of 
the  experimentation;  it  is  felt  that  the  improved  results  would  justify  any 
additional  expense.  Lastly,  from  the  operations  research  analyst's  point  of 
view,  there  is  a need  for  further  information  regarding  the  proficiency  and 
training  of  the  operators  involved  in  the  testing.  It  was  felt  that  further 
information  could  have  been  acquired  had  the  experimentor  known  to  what 
degree  the  subjects  were  random,  or  if  their  performance  was  truly  indicative 
of  the  general  class  of  communications  operators  which  would  operate  this 
type  of  equipment. 
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IME-SER I ES  MODELING  OF  MAN/MACHINE  INTERFACES 


2.1  INTRODUCTION 

'The  best  non-linear  regression  model  which  characterizes  the  relation- 
ship between  me  dependent  and  independent  variables  found  in  Section  1, 
dealt  with  the  prediction  of  the  number  of  committed  errors  as  a function  of 
ambient  light  and  acoustic  noise.  More  often,  the  communications  engineer 
is  interested  in  such  factors  as  reliability,  performance,  and  efficiency  as 
a function  of  time.  Thus,  utilizing  time-series  models,  it  may  be  possible 
to  characterize  each  operator,  or  take  the  group  as  a whole  for  predicting 
in  near-^ea 1 -time  the  number  of  committed  errors  at  times  t.,  , t0 , t,,  ..., 
t^,  in  the  future.  The  time-series  approach  for  this  type  of  information  is 
somewhat  unique  in  that  this  is  one  of  the  first  attempts  to  implement  this 
methodology  in  analyzing  cime-aeoendent  man-machine  interface  data.  In  view 
of  this  uni oueness,  there  are  a number  of  shortcomings  that  were  experienced. 
One  of  the  most  serious  limitations  was  the  sample  size.  However,  enough 
information  is  available  so  that  one  can  iniriave  the  time-series  methodology 
into  this  particular  subject  area.  This  approach  is  extremely  useful  because 
it  characterizes,  within  reason,  the  error  performance  of  any  communications 
terminal  eouipment  operator  working  in  a tactical  environment. 

Incorporated  into  the  design  of  the  experiment  ^detailed  in  Section  1) 
was  a four-second  time  interval  counter.  This  provided  a running  count  of 
the  number  of  transcribed  errors  in  each  four-secor.d  time  period  for  the 
duration  of  the  test.  Thus,  thirty-two  non-deterministic  time-series  were 
created  (sixteen  per  terminal , one  corresoondi ng  to  each  combination  of 
^nvi ronmental  factors).  Of  the  time  seizes  so  obtained,  and  because  of  the 
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In  section  2.3,  the  appropriate  forecasting  models  will  be  developed  to 
predict  the  number  of  errors  committed  by  the  operators.  This  section  will 
include  filtering  the  data  to  eliminate  non-stationarities,  model  identifica- 


tion, estimation  of  the  appropriate  parameters,  and  diagnostic  checking. 
Actual  steps  ahead  forecasting  and  updating  the  future  values  of  operator 
performance  will  be  addressed  in  section  2.4.  Finally,  a summary  and  conclu- 
sions are  presented  in  section  2.5. 


2.2  SOME  BASIC  CONCEPTS  OF  TIME-SERIES  MODELING 
Identifying  the  Stochastic  Realization 


In  a given  physical  situation  such  as  the  man-machine  interface 
experiment,  we  have  available  a time  series,  say  x^,  Xp,...,  xn,  of  n obser- 
vations. Our  aim  is  to  obtain  a suitable  difference  equation  or  model  that 
will  accurately  represent  the  true  underlying  process  which  generated  the 
information,  x^.,  t=l,2,...,n.  First,  we  must  identify  whether  the  series  xt 
exhibits  stationary  or  non-stationary  properties.  That  is,  when  we  speak  of 
stationary  time  series,  we  imply  that  the  statistical  properties  of  the 
series  are  independent  of  absolute  time.  A graphical  representation  of  the 
experimental  data  would  be  of  some  aid  in  exercising  judgment  about  the 
behavior  of  the  data.  Of  greater  importance  is  the  sample  autocorrel ation 
function  given  by: 

- > cxx(k) 

rxx(k)  = r ’ k = 0,1,2,... ,n-l 

cxxiu; 


where  cxx(k)  is  the  sample  autocovariance  function  defined  by: 

n-k 

cxx(k)  = n Z (xt  ‘ x)  (xt+k“*)’  k=0,l ,2, . . .n-1 . 
t 1 


of  the  observed  data.  If  the  observations  are  stationary,  the  sample  auto- 
correlation function  would  exhibit  fairly  rapid  dampening.  Furthermore,  one 
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can  apply  variins  c.  tat  i stical  tost.'-,  to  cherf  rnr  non-stationarv  properties. 

We  have  used  Kendall's  tau  test  [7]. 

If  the  man-machine  interface  data  were  not  in  statistical  equilib- 
rium (stationary ! , then  we  can  filter  out  the  non-stationary  components  by 
usinq  various  difference  filters.  In  all  environmental  combinations  of  the 
man-machine  interface  experiment,  the  resulting  lata  was  shown  to  contain 
non-stat ionary  components . 

P ' neneral  difference  filter  is  qiven  by: 
yt  = ( 1-B )d  x t , 

where  R is  a shift  operator  and  d is  the  order  of  the  filter.  When  d=0,  this 
will  indicate  that  the  experimental  data  is  stationary;  d=l,  will  indicate 
that  a first  difference  filter  is  necessary  to  filter  the  oriqinal  series, 
and  so  on.  For  the  man-machine  information,  we  used  filters  up  to  d = 2. 

The  procedure  to  determine  the  proper  value  for  d is  to  compute 
the  first  differences  of  the  oriqinal  data,  xt,  t=l,2,...,n.  That  is,  we 
process  Xj.  through  a first  difference  filter: 

Yf  = (1-B)  xt  = xt  - xt_j  , 

which  will  have  (n-1)  observations,  and  then  through  a second  difference 
f i 1 ter : 

wf  - (1-B)“xr  = xf  - 2x4-_ -j  + xt_,. 

which  will  have  (n-2)  observations.  for  the  oriqinal  observations,  xt,  and 
the  filtered  yt  and  w.,  we  calculate  the  sample  autocorrelation  function  and 
Kendall's  tau  . By  observing  the  sample  autocorrelation  function  of  the 
oriqinal  series,  the  filtered  i nformation, and  the  results  of  the  trend  test 
(Kendall's  tau:,  we  can  infer  a suitable  value  for  d,  that  is,  the  deqree  of 
'diffe-ensinq  necessary  to  induce  the  sample  autocorrelation  function  to 
dampen  out  fairly  rapidly  and  cause  endall's  tau  test  not  to  be  significant. 


42 


For  predicting  or  forecasting  operator  performance  for  one  or  more 


time  slots  in  advance,  the  initial  step  is  to  determine  the  particular 
process  that  characterizes  our  data.  There  are  three  basic  models  that  are 
candidates  for  this  purpose: 

a.  The  Autoregressive  Process 

b.  The  Moving  Average  Process 

c.  The  Mixed  Autoregressive-Moving  Average  Process. 

A discrete  m-order  autoregressive  (AR)  model  is  of  the  form: 

xt"u  = 01 1 (xt-l‘u)  + a2^xt-2"u)  + **•  + am(xt-m"u)  +Zt  (2-2-1) 
where  x^.  is  the  autoregressive  series  which  is  being  generated  by  the  series 
Zt,  a purely  random  process,  o^,  ...,  am  are  the  parameters  of  the  non- 
ordered  process,  and  u is  the  expected  value  of  the  series.  Such  a process 

assumes  that  the  current  value,  x^.,  of  the  data  has  resulted  from  a linear 

sum  of  past  values  of  the  series,  together  with  an  independent  error  term, 

Zt,  not  connected  with  the  past. 

A discrete  q-order  moving  average  (MA)  process  is  given  by: 

xt  - u - Zt  - ' izt- 1 “ ' 2Zt-2  _ •**  “ 2qZt-q  * (2.2.2) 

This  process  is  a weighted  sum  of  a random  series,  Zt.  Each  realization 

(committed  error,  Xj.)  is  made  linearly  dependent  on  a Zt  and  on  one  or  more 

previous  Z 1 s . Also,  u is  the  expected  value  of  xt,  and  8j,  8^,  ....  are 

the  parameters  of  the  model. 

The  mixed  model  consists  of  the  autoregress i ve  and  moving  average 
model  where  m is  independent  of  q.  (m  is  the  order  of  the  AR  process  and  q 
is  the  order  of  the  MA  process). 

We  shall  discuss  in  some  detail  a procedural  approach  in  fitting  an 
autoregressive  model  to  the  man-machine  interface  series.  A similar  approach 
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can  hr  f ! lowed  to  form. late  the  proc-'1  dur*  for  the  moving  averaoe  and  m i xed 
process  with  minor  changes. 

"he  A itoreqressi ye  Process 

The  autoregressive  model  previously  defined  can  be  adapted  to 
represent  the  characterization  of  operator  performance  for  the  purpose  of 
forecasting.  Discussing  the  theory  of  the  general  m^  order  model  is  quite 
complicated  and,  therefore,  we  shall  first  give  a brief  discussion  of  the 
second  order  model  which  is  Quite  useful  in  many  physical  situations. 

The  second  order  discrete  autoreqressi ve  process  may  be  written  as: 


* u - + l2  (xf_p~u)  + Zt. 


(2.2.3) 


If.  at  the  initial  stage,  it  was  necessary  to  filter  the  data  to  have  the 
information  in  statistical  equilibrium,  ‘hen  we  muss  place  certain  restric- 
tions on  estimating  the  parameters  of  the  model  to  make  sure  that  our  series 
remains  stationary.  To  obtain  these  restrictions  on  the  parameters,  we  use 
the  concept  of  ? -transforms,  , to  obtain  the  characteristic  equation  of 
the  process.  Solving  the  characteristic  equation,  we  can  place  conditions  on 
its  roots  so  that  the  filtered  model  will  not  violate  the  assumption  of 
stationari ty. 

The  Z -transform  of  equation  ''..2.3)  is  given  by: 

( I - < i ? J - tpZ  ‘ ' • • - ‘ .’*■  , 

X L. 

and  its  transfer  function  H(?-1)  is  given  by: 


H(Z_1)  = 


( 1 ~ 1 1 Z “ 1 - 


(2.2.4) 


Thus,  the  characteristic  equation  of  the  second-order  autoregressive  model  is: 
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whose  roots  are  given  by: 


v/rx  -j  ^+4ao  a,  - /ii  ^+4ao 

-r a"d  ' -1 — r — - 


In  order  for  the  second-order  model  to  be  stationary,  we  must  restrict  the 
estimates  of  the  parameters  and  so  that  the  roots  of  equation  (2.2.4) 
will  be  contained  within  a unit  circle,  that  is,  | SjJ  and  l^l  must  be  less 
than  one.  This  is  equivalent  to  havinq  m,  and  oto  lie  in  a triangular  region 
formed  by  < 1,  < anc*  ”1  < < 1*  For  additional  details  see 

[l5]  , and  ^16j  . 

A similar  approach  can  be  carried  out  by  considering  models  of 
higher  order.  The  I -transform  of  the  m1-^  order  autoregressive  (2.2.1) 
process  is  given  by: 

(l-ajZ-1-^*2-  ...  - a ml~m)  (xt-u)  = Zt.  (2.2.5) 

The  transfer  function  of  (2.2.5)  is  of  the  form: 


H(Z-1 ) = . 

(l-ajZ-i-ajZ-2-  ...-  OmZ"m) 

The  characteristic  equation  of  the  m^*1  order  process  is  given  by: 

. ..  - Ojh  = 0 . 


(2.2.6) 


(2.2.7) 


Thus,  for  the  general  finite  autoregressive  model  to  be  in  statistical 
equilibrium,  we  must  estimate  the  parameters  of  the  process  so  that  the  roots 
of  equation  (2.2.7)  must  lie  within  a unit  circle. 


The  Fitting  Procedure 

The  initial  stage  in  developing  any  one  of  the  three  models  under 
consideration  usually  involves  deciding  the  order,  m,  of  the  model,  and  then, 
given  m,  estimating  the  parameters,  u,  a?,  ...,  tm. 
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The  criterion  tor  se1 acting  the  best  order  which  characterizes  the 
observed  series  is  based  upon  the  residual  variance.  We  proceed  by  estima- 
tion ai  '•*- >-  ‘ the  m <de  • +,,r  different  orders,  and  then  the  residual 

variances  are  computed  and  plotted  against  the  order  of  to,  p-oc'-ss.  ‘he 
ininimum  residual  variance  will  correspond  to  the  order  of  the  model  which 
host  or  ■ r'p'-s  • •'•xp^rtroen-al  data  Thus,  for  the  autoregressive  process, 
it  is  necessary  to  first  studv  the  estimation  of  the  parameters  of  the  me lei . 

To  estimate  the  parameters  of  this  process,  we  can  use  the  method 
or  maximum  likelihood.  We  assume  that  the  Z+  process  is  normal.  Then,  for 
a fixed  m,  the  joint  probability  density  function  of  the  variates,  Zri|+], 
is  given  by: 
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v.  r ’»•  o;  of  .robabil'f  density  function  of  \ , x„  copdiHpn--»1 

on  Xj=Xj,  X2=X2»...»  nti'-”  to  obtain  the  joint  probability  density 

function  of  X^.  X?,  ....  Xn,  it  is  only  necessary  to  multiply  eouation 
iZ.2.°)  with  the  density  of  >', , ....  Xm.  Since  m,  xnr  practical  apo1  i- 

nations.  is  usually  small,  the  net  effect  of  not  carrying  out  this  mult  - 
. : ;af f o.|  small  a»>d  will  be  omitted,  for  details.  r*r  [lb]. 


•if 


The  Log-likelihood  function  of  the  process  may  be  written  ? 


Since  xj,  x^,  and  X3  are  usually  close  to  the  overall  mean,  K,  we  can  use  it 
as  an  approximate  estimate  of  u.  Furthermore,  we  can  obtain  good  approximate 
estimates  of  (a)  through  (c)  using  the  sample  autocorrelation  function  at  lag 
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(2.2.11) 


one,  rxx(l). 

That  is, 

cxx(l)  - 

“1  cxx(°)  + 

i2  Cxx(l)  , 

and 

A 

’1  Cxx(l)  + 

■2  S*«»  ■ 

The  autocovariance  is  an  even  function,  thus,  we  can  write  equation  (2.2.11) 
as  follows: 

cxx(j)  ~ al  cxx^"1)  + *2  cxx(j"2)  . J = 1.2  . (2.2.12) 

An  approximate  estimate  of  the  parameters  ^ and  i2  is  given  by: 


and 


(2.2.13) 


- „ rxx(D[l-rxx(2)] 

11  i-4(d 

- _ rxx(2)-rxx(D 

2 1-r2  (1 ) 

a A 

Also,  an  estimate  of  the  residual  sum  of  squares  can  be  obtained  in  terms  of 
the  sample  autocovariance  function.  That  is: 


A A A 


S(:-.''l.«2>  * ("-2)  <«„<«>- *»«<*>' 

and  the  residual  variance  of  Zt  is  given  by: 


(2.2.14) 


S2  = J- 
z n-5 


S(u,  a.j,  ou), 


Similar  expressions  can  be  obtained  for  estimating  the  parameters  for  the 
general  finite  autoregressive  model.  The  normal  equations  may  be  approxi- 
mated by  using  the  sample  autocovariance  given  by: 

cxx(j)  ~ ^icxx(J_1)  + ^2cxx('i-2)  •••  + ^mcxx('i“m)  (2.2.15) 

j = 1,  2,  ...,  m.  Approximate  estimates  can  be  obtained  for  the  parameters 
aj,  ou,  ...,  by  solving  the  m simultaneous  equations  (2.2.15). 

The  residual  sum  of  squares  and  the  residual  variance  may  be 
obtained  by  using  the  following  approximations : 
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and 


S(y,  * • • • •'-»m ) ~ ( n-m)  ^c^^(0)-ot2C^^(l)  ~ • • • ~oc^c^^ ( m ) / , (2.2,15a) 

c2  - _L 


n-2m-l 

respecti vel y. 


S(u,  'j,...,  jtm)  , 


(2.2.15b) 


Checking  the  Fit  of  the  Model 

Once  we  have  selected  the  best  process  that  characterizes  the  man- 
machine  interface  data  and  have  its  pirameters  estimated,  diagnostic  checks 
are  made  on  the  model  to  determine  its  adequacy.  Using  this  model,  we  can 
obtain  a series  that  should  simulate  the  behavior  of  the  original  information. 
If  the  original  information  were  filtered,  that  is,  d was  different  from 
zero,  it  would  now  be  necessary  to  use  a "backwards  filter",  replacing  yt  in 
the  model  with  (l-B)^x^,  and  using  the  resulting  process  to  forecast  operator 
performance.  For  example,  if  we  fitted  a first-order  autoregressive  model: 

yt-i:  = ot1  (y^-pu)  + Zt  , (2.2.16) 


where  yt  = (1-B)xt  = xpx^  is  the  Filter  used  in  the  original  data,  then 
inserting  the  filter  into  (2.2.16),  we  have: 

xt  = *o  + X1  xt-l  + x2  xt-2  + Zt  • 

where  x0  = 2(1-^)  , ooj  = (l+r*)  and  x2  = -Sj.  Thus,  the  mth  order  auto- 
regressive process,  using  a first  difference  filter,  can  be  written  as 
fol 1 ows : 


xt  1 o 1 1 xt-l  + 1 2 Xt-2  + + Tm+d  xt-m-d  + Zt’  (2.2.17) 

where  the  values  of  7^1'  = 1,  2,  ...,  m+d,  will  depend  on  n and  d. 

For  the  fitted  model  to  give  a good  characterization  of  operator 

A 

performance,  the  residuals,  rt  = xpx^.,  t = 1,  2,  ...,  n,  should  behave 
approximately  like  random  deviates.  Hence,  the  sample  autocorrelation  func- 
tion should  effectively  be  zero  for  all  lags  except  the  zero^-h  lag. 
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Forecasting  and  Updating  the,  1'odel 

One  of  the  aims  in  having  fitted  an  autoreqressi ve  process  to  the 


experimental  data,  is  to  forecast  f iture  values  of  t.hn  committed  errors 
(operator  performance).  If  we  wish  to  forecast  a particular  puce  of  infer- 
mation,  x^.+  , i,  when  we  are  presently  at  time  slot  t,  then  the  forecast 

is  made  at  origin  t ‘‘nr  a lead-time  . nr  course,  the  shorter  the  lead-time 
, the  more  accurate  our  forecasted  value  will  h«~ . 

The  minimum  mean  sou a re  error  forecast  for  any  lead  time  is  given 
bv  the  conditional  expectation,  [15]  , , , of  x*+  , at  time  slot 

(origin)  t,  given  knowledge  of  all  x's  up  to  time  t.  That  is: 

f xt+?3  = x t ( 

Replacing  t with  t ‘ in  equation  (2.2.17),  we  have: 

xt+  = :r,o  + Dixt+.  + ' 2xt+  -2  + •••  + 1 m+dxtJ-- -m-d  + Zt-*-  . 

The  minimum  mean  square  error  forecast  of  the  ionospheric  data  is  g i ver-  by: 


" ;,  a non-nenative  integer,  we  know,  [lb;,  that: 


t+jJ 


x + ( j i,  E<  J-v-'i 


and 


Et  LXt-jJ  xt-j’  EtiZt-jj  = Zt-j  = xt-j'xt-j-l  • (2.2.20) 

Therefore,  we  con  write  equation  (2.2.18)  as  follows: 

xtv  - i n 1 1 1 Xfj-  _ i + •••  + 1 m+d  xt+  -m-d  * (^.2.21) 

The  variance  of  the  ■ step  ahead  forecast-error  for  any  time  slot  t,  is  the 
expected  value  of: 


2, 


v m 


uxt-.  ~ At> 


i T 2 


(2.2.22) 
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Box  and  Jenkins,  [16],  have  shown  that  the  variance  of  the  lead  time,  £,  is 
given  by: 

£- 1 P p 

Var  (£)  = {1  + Z 0-  } (2.2.23) 

j = l J 

where  a\  is  estimated  by  s|,  that  is: 

A A A 

p _ Siu.ot!,...^) 


and  0j  is  given  by: 


6j  = 0,  j<0 
0O  = 1 
91  =QPl 

02  = CDj^  0J+  qp2 


6j  = 03 1 0 j - 1 + **•  +QOm+d  ej-m-d  * 

The  (l-a)%  confidence  limits  for  xt+£  is  given  by: 


(2.2.24) 


Pr  (xt(£)-Ua(l+Z  6j)"2  sz  < xtH  < xt(£)+Ua(l+I  0?)*Ssz  =1  -a 

2 J=1  ' 7 1 = 1 

where  U is  the  deviate  from  the  unit  normal  probability  distribution. 


In  man-machine  interface  problems,  we  are  often  interested  in 
forecasting  future  values  of  an  observed  series  for  several  time  slots  in 
advance.  When  we  forecast  values  at  leads  greater  than  or  equal  to  two  (£>2) 
with  an  autoregressive  process,  the  forecasted  value  will  be  dependent  on 
previously  forecasted  values;  but,  as  additional  data  becomes  available,  we 
can  update  our  old  forecast  by: 

Xt+1  = xt(^+l)  + 0£^t+l  * 


That  is,  the  "t"  origin  forecast  of  x^+  +|  can  be  updated  to  become  the 
"t  + 1"  origin  forecast  of  the  same  value,  xt+  +j,  by  adding  a constant 

multiple  of  the  one-step  ahead  forecast  error  Z,+1,  where 

Zt+1  = Xt+1  ‘ xt^ ' J 
is  used  with  multiplier  6„. 

2.3  FORECASTING  MODELS  FOR  CHARACTFPIZKIG  MAN-MACHINE  IilTFRFACFS 

The  objective  of  this  section  is  to  utilize  both  non-stationary  and 
stationary  data,  generated  by  the  experiment  outlined  in  section  1.2,  to 
develop  models  for  predictina  communications  operator  performance.  More 
precisely,  we  shall  fit  the  appropriate  model,  either  AR,  MA,  or  ARMA,  to 
man-machine  performance  data  obtained  for  two  communications  terminal  eguip- 
ments.  The  series  involved  will  be  those  for  a deteriorating  environment 
1,4-  and  the  most  deteriorated  environment  {4,4 } as  specified  in  table  1.1. 
This  involves  a detailed  analysis  of  four  observed  series  where  the  discrete 
intervals  have  a duration  of  four  seconds.  Thus,  we  are  concerned  with  the 
response  variable,  namely,  the  number  of  committed  errors  pe>'  four-second 
interval,  as  a function  of  the  transcription  time  of  a message  (refer  to 
section  1.2). 

2.3.1  Identifying  the  Series 

To  determine  if  the  series  for  the  standard  teletypewriter  terminal 
(TTY)  and  the  Optical  Display  Terminal  (ODT)  exhibit  stationary  or  non- 
stationary  properties,  one  must  first  try  to  visually  detect  any  trend  or 
non-randomness.  Figures  2.1,  2.2,  2.3,  and  2.4  are  plots  of  the  data, 

x*,  t = 1,  2,  n,  where  n 50.  The  visual  interpretation  of  the  series 

appears  to  indicate,  for  the  environmental  combinations  {1,4}  and  (4,4',  that 
non-stationari ties  do  exist.  In  all  cases,  however,  this  is  not  true,  as  we 
will  now  proceed  to  verify.  It  is  to  be  again  emphasized  that  visual  inter- 
pretation must  be  validated  by  using  graphs  of  the  autocorrelation  functions 
and  statistical  tests  for  trend. 

Figures  2.5,  2.6,  2„7  and  2.8  show  the  sample  autocorrel ation  functions 
for  the  observed  data.  Figures  2.5  and  2.8  are  excellent  examples  of  rapid 
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GURE  2 .2  COMMITTED  ERRORS  HER  FOUR  SECOND  INTERVAL 
OPTICAL  DISPLAY  TERMINAL  FOR  THE  1, 


ID  ERRORS  PER  FOUR  SECOND  INTERVAL  OBSERVED  A 
ETYPEWRITER  TERMINAL  FOR  THE  4,  4 ENVIRONMENT 


uoLiP^-woscnnv  aiduips 
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1.6  SAMPLE  AUTOCORRELATION  OF  THE  COMMITTED  ERRORS  PER  FOUR  SECOND 
INTERVAL  OBSERVED  AT  THE  OPTICAL  DISPLAY  TERMINAL  FOR  THE  1 , 4 ENVIRONMENT 


FIGURE  2./  SAMPLE  AUTOCORRELATION  OF  1HE  COMMUTED  ERRORS  PER  FOUR  SECOND  INTERVAL  OBSERVED 

AT  THE  TELETYPEWRITER  TERMINAL  FOR  THE  4,  4 ENVIRONMINT 


FIGURE  2.8  SAMPLE  AUTOCORREl AI ION  OF  THE  COMMITTED  ERRORS  PER  FOUR  SECOND  INTERVAL 
OBSERVED  AT  THE  OPTICAL  DISPLAY  TERMINAL  FOR  THE  4,  4 ENVIRONMENT 


— — 1 

dampening  and  indicate  that  the  {1,4}  environment  (TTY  data),  and  the  {4,4} 
environment  (ODT  data),  are  stationary  realizations  requiring  no  filtering 
of  the  information.  As  to  the  contrary,  figures  2 -6  and  2.7  show  some 
dampening,  but  due  to  large  peaks  between  time  slots  10  through  20,  the  inai- 
cation  is  that  some  form  of  filtering  is  required.  Figures  2.9  and  2.10  show 
the  sample  autocorrelations  of  the  first  filtered  information  for  the  '1,4'. 

ODT,  and  {4,4}  TTY.  Dampening  in  these  cases  has  obviously  been  improved, 
especially  in  the  case  of  the  {1,4}  ODT  data.  Still,  however,  large  peaxs 

■L.  L.  M.  u 

remain  in  figure  2.10  between  the  10  through  20z  time  slots.  With  the 
indication  that  the  {1,4}  ODT  data  is  now  stationary,  the  sample  autocorrela- 
tion of  the  second  differenced  {4,4}  TTY  data  was  plotted  (see  figure  2.11;. 
Dampening  in  this  case  is  again  improved  about  the  zero  axis  due  primarily 
to  the  increased  peak  at  the  second  time  slot.  New  one  can  surmise  that  the 
second  differenced  data  is  in  statistical  equilibrium. 

In  each  case,  Kendall's  Tau  test  was  performed  on  the  filtered  informa- 
tion (the  {1,4}  ODT  and  the  {4,4}  TTY  data)  and  on  the  jnfiltered  information 
(the  {1,4}  TTY,  and  the  {4,4}  ODT  data),  to  verify  the  indications  shown  by 
the  graphs  of  the  autocorrelation  functions.  Table  2.1  shows  the  results  of 
the  trend  tests. 


Table  2.1  Kendall's  Tau  Test  for  the 
Man-Machine  Interface  Series 


SERIES 

1 

| 

CALCULATED 

STATISTICS 

■OBSERVED 
. SERIES 

FIRST  DIFFERENCE 
DATA 

SECOND  DIFFERENCE 
DATA 

REFERENCE 

7 

- 25 

{1,4}  TTY 

-0.520 

- 1 .545 

{1  ,4}  ODT 

2.656 

0.979 

z 1 .645 

{4,4}  TTY 

2.477 

2.328 

1.089 

;1 . 545 

{4,4}  ODT 

0.817 

; 1 .545 

Clearly,  in  comparing  the  calculated  statistics  to  tne  aQorcDriate  reference 
at  the  a - .05  level  of  significance,  the  indications  of  the  sample  autocor- 
relation functions  are  verified.  Therefore,  one  must  conclude: 

a.  that  the  {1,4}  ODT  data  is  stationary  after  a first  difference 
filter  is  used. 


i 

J 
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b.  that  the  (1,4}  TTY  observed  data  and  the  {4,4}  ODT  observed  data  are 
in  statistical  equilibrium  and  no  filtering  is  required,  and, 

c.  that  the  {4,4}  TTY  data  requires  a second  difference  filter  to 
remove  the  non-stationarities. 

2.3.2  Fitting  the  Models  for  Man-Machine  Interface  Data 

To  fit  one  of  the  stationary  stochastic  models  discussed,  namely,  the 
autoregressive,  moving  average,  and  a mixture  of  the  two,  the  parameters  are 
estimated  for  various  orders  (m,q),  taking  into  consideration  the  restric- 
tions on  the  parameters  to  insure  stationarity  and/or  the  invertibil ity  of 
the  stochastic  process.  Simultaneously,  the  residual  variances  are  computed 
for  each  of  the  orders  under  consideration.  Fi gures  2 . 1 2 , 2 . 1 3,  2.14,  and 
2.T5  show  the  calculated  residual  variances  as  a function  of  order,  (m,q). 

We  use  the  criterion  of  minimum  residual  variance  to  select  the  appropriate 
model.  If  there  are  competitive  residual  variances,  then  we  select  the 
model  of  least  order  to  avoid  complicated  handling  of  the  data.  This  is 
especially  important  for  field  implementation  of  the  forecasting  scheme  which 
can  be  accomplished  with  a hand-held  calculator,  given  the  appropriate 
parameter  values.  Thus,  from  the  above  figures,  the  following  models  were 
selected: 

i)  (2,3)  ARMA  process  for  the  teletypewriter  terminal,  {1,4}  environ- 
ment, 

ii)  (3,0)  AR  process  for  the  optical  display  terminal,  {1,4}  environ- 
ment, 

iii)  (1,3)  ARMA  process  for  the  teletypewriter  terminal,  {4,4}  environ- 
ment, and 

iv)  (0,3)  MA  process  for  the  optical  display  terminal,  {4,4'.  environ- 
ment. 

In  subsequent  sections,  it  will  be  shown  that  the  above  model  identifications 
are  the  most  appropriate  ones  to  characterize  the  data. 

With  the  order  of  the  process  determined,  the  associated  parameters 
which  were  simultaneously  computed  are  now  also  known.  Table  2.2  shows  these 
estimates  for  the  appropriately  filtered  series. 
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Taple  2.2  Approximate  ^east  Squares  Estimates 
of  the  Best  Model  Parameters 


MODEL 

/ SERIES  / 

ORDER 

u 

*1 

a2  a3 

31 

A i 

AZ  3 

i . 

{1,4}  TTY 

(2,3) 

1.699 

0.660 

0.367  

-0.449 

-0.223  -0.422 

i i . 

{1,4}  0DT 

(3,0) 

0.000 

-0.746 

-0.613  -0.258 



i i i . 

{4,4}  TTY 

(1,3) 

0.028 

-0.215 



-0.957 

-0.191  -0.016 

i v . 

•.4,4}  00T 

(0,3) 

2.158 



0.453 

-0.023  -0.051 

Note 

that  in  all 

cases. 

the  estimated 

parameters  sati 

sfy  the 

stationari ty 

ano/'or  invertibi 

1 ity  conditions 

These  resul ts  yield 

, respectively,  the 

following  difference  equations  in  terms  of  either  zero,  first,  or  second 
difference  filters  as  established  in  section  2.3.1: 


i . 


■ 1 i . 


in. 


■ v . 


( x t - 1.699)  = 0.660  (x 


t-1 

0.223Z. 


- 1.699)  + 0.367  (xt_2  - 1.699)  + Z. 

- O.A49Zt_1  + 0. 223Zt_2  - 0.422Zt_3  (2.3.1) 

(yt  - 0.000  = -0.746  (y  -0.000)  - 0.613  (yt_2  - 0.000) 

- 0.258  (yt_3  - 0.000)  + Z,  (2.3.2) 

0.191Z. 

(2.3.3) 


K - 0.028)  = -0.215  (w*_1  - 0.023)  - Z.  * 0.957Zt_1  - o..3.is2 


- 0 . 01 6Z* _ 3 

(xt  - 2.158)  = Zt  - 0.453Zt_1  + 0.023Zt_2  + 


0. 051 Z 


t-3 


(2.3.4) 


2.3.3  Inserting  The  Backwards  Filter  And  Diagnostic  Check  Of  The  Models 

To  determine  the  adeauacy  of  the  fitted  processes,  the  observed  series 
must  be  simulated,  and  then  the  residuals  must  be  calculated  to  see  if  they 
behave  as  a purely  random  process.  Recall,  that  in  order  to  use  equations 
2.3.1,  2.3.2,  2.3.3,  and  2.3.4  to  simulate  the  observed  series,  xt,  we  make  use 
of  the  backwards  filter  which  depends  upon  the  original  filter  employed  to 
transform  the  original  series.  Referring  back  to  table  2.1,  we  must  employ 
the  backwards  filters  to  equations  2.3.2  and  2.3.3.  Equations  2.3.1  and  2.3.4 
required  no  filtering  prior  to  fitting  the  data.  Hence,  inserting  the 

appropriate  filters  into  eauations  2.3.2  and  2.3.3,  the  following  characteri- 
zations are  obtained  for  simulation  of  man-machine  series: 
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1 


i . 

xt  = 

-0.046  + 0 

. S60x*  , * 0 
0"  1 

.367x^  7 

^ Zt  +0-449Zt-l 

+ 0. 223Z*  - 

L 

* 0. 422Z„_ 

3 

(2.3.5) 

i i . 

xt  = 

-0.254x*_1 

+ 0.1  33xt._2 

+ 0.355x 

t-3  4‘  °-258xt-4 

+ Zt  (2.3.6) 

i i i . 

xt  = 

0.006  * 1. 

785xt_^  - 0. 

570Xj.  2 ' 

0.215xt_3  + Zt 

+0. 957Z*  , 

u-  1 

0. 1 91 Zt_2 

+ 0.01 6Zt_3 

(2.3.7) 

iv. 

xt  = 

2.158  + Zt 

-0.453Zj._-j 

+ 0.023Zt 

.2  + 0.051  Zj._3 

(2.3.8) 

Setting  the  unknown  values  of  Z*  equal  to  their  unconditional  expectation  of 
zero,  we  begin  the  simulation  by  initially  assuming  x-j  , x are  known; 

to  simulate  x^.,  we  assume  x^  , xt_m_^,  are  known.  Figures  2.16,  2.17, 

2.13,  and  2.19  show  the  simulated  series  along  with  the  observed  series. 
Clearly,  we  have  a good  fit  of  the  estimated  models  with  respect  to  the 
observed  series. 

To  check  if  the  residuals  benave  as  a ourely  random  process,  the  sample 

autocorrelation  function  of  the  residuals,  r (k),  was  calculated  for  all 

1 / 

lags.  It  is  known  that  r (k)  ^ N(0,  /n).  The  calculated  value  of  the 
standard  deviation  of  r (k)  was  found  to  be  9.158,  and  the  95?  confidence 
limits  are  : (1.96)  (0.158)  = t 0.310.  At  the  5?  level  of  significance,  one 
could  expect  (.05)  40  or  2 out  of  the  sample  autocorrelations  to  lie  outside 
the  confidence  interval  for  the  {1,4}  environment.  The  calculated  confidence 
interval  for  the  {4.4}  environment  is  .347.  Taoles  2.3,  2.4,  2.5,  and  2.6 
show,  for  the  four  time  series  considered,  that  few  of  the  sample  autocorrel- 
ations lie  outs'ae  the  confidence  limits. 


S’.  , 

Table  2.3  Sample  Autocorrelation  of  the  Residuals,  r (k),  for  the 


Simulated 

{1  ,4- 

TTY  Model . 

Confidence 

Interval  = t 

0.310 

Lags 

Sample  Au 

tocorrel ati 

: on 

of  Residuals 

1-3  ! 

1.00 

.30 

.52 

.51 

,48 

.55 

.53 

.52 

9-’5 

.43 

• . J 1 

.29 

.32 

,36 

.32 

.24 

.17 

17-24 

.10 

.23 

. 28 

25 

.19 

.15 

.19 

25-32  | 

.25 

.25 

.30 

.27 

23 

.20 

.21 

.25 

33-40  ! 

.25 

. 23 

.20 

.17 

17 

.14 

.11 

.06 
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Fll  It RED  MODEL: 
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Eour  Second  Time  Intervals 

FIGURE  2.17  SIMIJIATED  MAN/MACHINE  INTERFACE  SERIES  USING  THE  AUTOREGRESSIVE  MODEL  vs.  THE 
OBSERVED  INFORMATION  FOR  THE  OPTICAL  DISPLAY  TERMINAL,  1,4  ENVIRONMENT 


m it  id  u Monti : 
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able  2.4  Sample  Autocorrelation  of  the  Residuals,  r (k),  for  the 
Simulated  {1,4’-  0D7  Model.  Confidence  Interval  = t 0.310 


Lags 

Samoie 

Autocorrelation  of 

Res 'dual s 

1 -8  j 1 . 000 

.602 

.362 

.122 

.008 

-.012 

.010 

-.006 

9-16  ; -.001 

-.008 

.005 

.000 

.001 

.011 

.018 

.005 

17-24  i .020 

.023 

.012 

-.011 

-.009 

-.008  - 

.017 

-.020 

25-32  .009 

.001 

- . 007 

-.004 

-.005 

.003 

.010 

.005 

33-40  . 3C2 

-.000 

.013 

.053 

. 1 29 

. 1 56 

.218 

.219 

Table  2.5  Sample  Autocorrelation  of  the  Residuals,  r (k),  for  the 
Simulated  -4. a'-  TTY  Model.  Confidence  Interval  = * 0.347 

Lags  Samoie  Autocorrelation  of  Residuals 


1-3  , 

o 

o 

o 

-.030 

- . 044 

- . 004 

.002 

-.002 

-.000 

- . 002 

9-16  1 

-.007 

-.000 

- . 002 

.002 

-.003 

-.001 

-.014 

.002 

7-24 

.002 

-.012 

-.022 

.041 

- . 006 

.036 

-.012 

- . 01  5 

5-32 

-.009 

on  i 
. ■ j i 

.002 

- 01  - 

- . 009 

^0*1 

. 006 

~ n 

■ j - JO 

ml  a 

* . j i u 

- . 

-\  o p 

. 060 

'nn 

1 


Hence,  we  conclude  that  the  residuals  for  the  four  man-machine  interface 
cnaracteri nations  do  behave  as  a purely  random  process,  and  thus,  equations 
2.3.5,  2.3.6,  2.3.7,  and  2.3.8  give  satisfactory  representation  of  the 
observed  series. 


2.4  FORECASTING  AND  UPDATING 


To  forecast  "4"  time  slots  in  advance  of  any  origin,  t > m + d,  we 
’’ODlace  t with  t + 4 in  equations  2.3.5,  2.3.6,  2.3.7,  and  2.3.8.  Having 
made  the  apnropriate  substitution,  we  obtained  the  following  minimum  mean 
square  error  forecasting  models: 

i.  for  the  {1,4}  TTY  characterization: 

x*(  4)  = -0.046  + 0.660xi+,  , + 0.367x^.J_  - - 0.449Z1.  + , , 

+ Q.223Zt+4-2  + 0.422ZtH_3  , (2.4.1) 

for  the  -{1,4}  ODT  characterization: 

;<tU)  = - 3.254xt+i_,  + 0.133x^,_2  - 3.355x^._3  - 0.253xt+4_4  , 


i : 


in 


"or  the  {4,4} 


Y characterization: 


xt(4)  = 0.006  1 . 785x^^_^  ^ - 0.570x^.+^  ? - 0.215xk+,^  3 


+ 0.950Z, 


0.191Z, 


0. 016Z, 


-4-3 


’ 7 . 


.-4-1  u' ' J,I-t-t4-2 
for  the  {4,4}  ODT  characterization: 

- 2.153  - O.ASSZ^  * 0.023Ztn-2  - O.OSIZ^ 


(2.4.2) 

(2.4.3) 

(2.4.4) 


:n  brie1-',  suppose  that  at  t = 20.  a forecast  x^  is  made  for  the  t = 22  time 
slot.  x99  may  be  updated  to  become  the  t = 21  origin  forecast  of  x97  oy 
adding  a constant  multiple  of  the  one-step  a.nead  forecast  error,  3,Zt,,  = 
-.Z9,  , to  the  t = 20  origin  rorecast  of  x9„.  That  is:  x*9  = x90  * 3,Z91. 
“he  ^precast  er'-or  for  this  case  would  be:  = x^  - x^  and  3,  = 3-  is 

given  by  eauations  (2.2.19)  and  (2.2.20) . The  updating  is  done  wnen 
x*-*]  = x 2 ■>  becomes  available.  Tables  2-7,  2-3,  2-9,  and  2.10  show  the  L 
steps  ahead  forecasts  (up  to  4 = 11 ) at  an  arbitrary  t * 20  origin,  with 
updating,  along  with  their  95*  confidence  intervals. 


Table  2.7  Forecasted  Values  of  the  {1,4.}  TTY  Series  at  Origin  t = 2 
and  Updating  Under  the  Assumption  x.,.,  Becomes  Availaole 


AC i JAL 

LEAD 

35'  -3D0BA- 

UPDATE' 

ERRORS 

TIME 

FORECAST 

3I.:~V  -EMITS 

foreca: 

3.0 



_ _ 



5.0 

1 

^ _ TQQ 

— 

0 

■ o 

4 . vj 

- . joJ 

I . j5 

*7  o 

1 

^ /•> 

j . j 

J 

. i 

r . 4ow- 

. 

3 

J, 

. 0 

r . 637 

.oc 

2.0 

5 

2 . 6 

r .309 

Cm  • -J  ' 

A 

5 

. 5 

= .307 

— « 1 

2.0 

7 

3.1 

: .997 

2.39 

1.0 

1.5 

=1 .034 

- o c 

1 . 0 

9 

1.4 

r:  1 .166 

- 1 - 

1 . 0 

i ■«-/ 

i .3 

= 1 .247 

1.53 

.0 

1 1 

.3 

=1.325 

. zl 

-orecas 

Ddating 

ted  Values 
.nde’-  the  A 

of  the  -I  ,4 
ssumot'on 

Series  at  Or i 
Becomes  Aval' lac  1 

5 i n 7 - c 

40.  j A L 
ERRORS 

LEAD 
“I  ME 

- uRtoAS 

95  DRCBA- 
l :m :~5 

update: 

"Oreca: 

J . J 

4.0 

-- 

3 . 3 

r .405 

— 

0.0 

A 

10. 1 

- .412 

? . 39 

.0 

3 

. 0 

= .4^7 

.30 

2.0 

4 

i .9 

= .486 

2 . 07 

3.0 

5 

3.0 

= .501 

3.16 

1.0 

5 

1 .2 

= .520 

1 . 03 

<rl_  Lii 


Table  2.9  Forecasted  Values  of  the  :4,4}  TTY  Series  at  Origin  t = 20, 
and  Updating  Undt  the  Assumption  Becomes  Available 


TIME 

INTERVAL 

ACTUAL 

ERRORS 

LEAD 

TIME 

FORECAST 

95%  PROBA- 
BILITY LIMITS 

UPDATED 

FQRECAS' 

20 

4.0 

-- 

— 



- 

21 

7.0 

1 

6.8 

± .297 

— 

22 

.0 

2 

.0 

± .752 

.090 

23 

1.0 

3 

1.1 

±1.335 

1.005 

24 

.0 

4 

.6 

±2.242 

.010 

25 

2.0 

5 

2.2 

±3.236 

2.100 

26 

.0 

6 

.0 

— 

.000 

27 

1.0 

7 

1.0 

— 

1.000 

28 

1.0 

8 

.9 

— 

.960 

29 

l.o. 

9 

1.0 

— 

1.000 

30 

2.0 

10 

1.9 

— 

2.100 

31 

1.0 

11 

1.4 

— 

.900 

Table  2 

.10  Forecasted 

Values  of  the  (4,4 

} ODT  Series  at  Origin  t = 20 

and 

Updating  Under 

the 

Assumption  X£.| 

Becomes  Available 

TIME 

ACTUAL 

LEAD 

95%  PROBA- 

UPDATED 

INTERVAL 

ERRORS 

TIME 

FORECAST 

BILITY  LIMITS 

FORECAST 

20 

2.0 

— 





_ 

21 

3.0 

1 

2.96 

r.275 

— 

22 

6.0 

2 

5.96 

±.273 

6.010 

23 

1.0  „ 

3 

.94 

±.233 

1.081 

24 

1.0 

4 

.94 

±.307 

1.174 

25 

1.0 

5 

.99 

±.307 

.976 

26 

1.0 

6 

1.07 

±.307 

1.025 

27 

.0 

7 

.03 

r . 307 

.000 

23 

2.0 

3 

2.07 

±.  307 

2.003 

29 

1.0 

9 

1.05 

±.  307 

. 973 

30 

1.0 

10 

1 .04 

±.307 

. 959 

31 

1.0 

11 

1.03 

±.307 

.940 
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Clearly,  the  forecasts  for  i < 11  are  quite  good  for  the  AR,  MA,  and  ARMA 
realizations.  For  the  ARMA  orocess  (-'1,4’  TTY  series,  table  2.7),  the  fore- 
casts for  /,  = 1,  2,  3,  are  not  very  good.  As  t is  increased,  the  variance 
and  the  confidence  limits  increase  correspondingly.  Also,  in  general,  one 
can  say  that  as  l increases  beyond  m + d (m  is  the  order  of  the  AR  compo- 
nent and  d is  the  order  of  the  filter),  the  forecasts  begin  to  over-estimate 
the  future  values.  This  fact  and  the  results  of  table  2.7  are  strongly 
indicative  of  the  need  for  updating  as  new  information  become  available.  As 
seen  in  the  tables  above,  accuracy  increases  with  updating. 

2.5  SUMMARY  AND  CONCLUSIONS 

The  procedure  developed  in  section  2.2  and  illustrated  in  detail  in 
sections  2.3  and  2.4,  was  followed  precisely  in  characterizing  the  man- 
machine  interface  data.  The  information  acquired  tnrough  an  extensive 
environmental  experiment  involved  two  different  communications  terminals  and 
six  experienced  communications-center  operators.  The  operators  were  tested 
under  sixteen  different  combinations  of  ambient  light  and  acoustic  noise 
(refer  to  section  1.2).  3ecause  of  the  extensive  data  base,  the  two  most 
critical  environmental  combinations,  namely,  (1,41  ana  (4,4,  were  adaressea 
in  this  treatise.  Clearly,  the  time-series  characterization  of  the  data  is 
very  promising  ~rpm  the  ooint  of  view  of  affording  to  the  communications 
system  designer  and  planner  a means  to  predict  the  human  element  of  the  total 
communications  system  architecture.  It  nas  been  shown  that  the  realizations 
obtained  were  very  adequate  in  characterizing  the  underlying  process  of  er~or 
oe^'ormance. 

In  review, 

a.  for  the  {1.4}  environment,  tglatupewi'dtev  ter”tir^zl,  we  obtained  the 
(2,3)  ARMA  process: 

x„  = -0.046  * 0. 660x„  , + 0.367x.  , A Zv  * 0.449Z*  , * Q.223Z.  - 

l.  U“  I l-c  U u-  t t-x 

+ 0.422Zt_3  ; 

b.  for  the  ,4}  environment,  optical  display  terminal,  the  (3,3)  AR 
process  obtained  was: 
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+ 0 . 1 33x 


x ^ 3 0.254x,._, 


t-2  * 3-355xt-3  + J,253xt-4  * Zt 


c.  *or  the  {4,4}  environment,  ssieourewri^sr  terminal,  we  obtained 
another  ARMA  orocess  of  order  (1,3): 

x.  = 0.006  + 1 . 785x  , - 0.570xt  , - 0.215x.  , * Z + 0.95QZ.  . 

+ 0.191  Z*.  , + 0.Q16Z,  , ; 

L-  u C“  J 

d.  and,  finally,  for  the  {4,4}  environment,  ovti 
the  (0,3)  MA  process  obtained  was: 

x't  = 2.153  + Zt  - 0.453Zt_1  * 0. 023Zt_3  - 0. 051  Z^_3 


ool  iisvlon  terminal. 


One  of  the  implied  features  of  this  chapter  is  that  for  each  environ- 
mental combination,  no  corner,  realization,  either  ARMA,  MA,  or  AR,  was 
obtained  to  characterize  operator  performance.  One  can  conclude,  therefore, 
that  even  with  an  adequately  developed  procedure  for  analysis,  more  than  one 
cnaracteri zation  is  required  to  evaluate  the  human  subsystem  in  sophisticated 
communications  systems,  "he  procedures  developed  in  section  2.2  clearly 
provide  a realistic  view  of  the  complex  man-machine  interface  that  occurs  in 
current  communications  systems. 
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